Tokenizing a string in C++

Tokenizing a string denotes splitting a string with respect to a delimiter. There are many ways we can tokenize a string. Let’s see each of them:

stringstream

A stringstream associates a string object with a stream allowing you to read from the string as if it were a stream.

Below is the C++ implementation :

// Tokenizing a string using stringstream 
#include <bits/stdc++.h> 
  
using namespace std; 
  
int main() 
{ 
      
    string line = "GeeksForGeeks is a must try"; 
      
    // Vector of string to save tokens 
    vector <string> tokens; 
      
    // stringstream class check1 
    stringstream check1(line); 
      
    string intermediate; 
      
    // Tokenizing w.r.t. space ' ' 
    while(getline(check1, intermediate, ' ')) 
    { 
        tokens.push_back(intermediate); 
    } 
      
    // Printing the token vector 
    for(int i = 0; i < tokens.size(); i++) 
        cout << tokens[i] << '
'; 
} 

Output:

GeeksForGeeks
is
a
must
try

strtok()

// Splits str[] according to given delimiters.
// and returns next token. It needs to be called
// in a loop to get all tokens. It returns NULL
// when there are no more tokens.
char * strtok(char str[], const char *delims);

Below is the C++ implementation :

// C/C++ program for splitting a string 
// using strtok() 
#include <stdio.h> 
#include <string.h> 
  
int main() 
{ 
    char str[] = "Geeks-for-Geeks"; 
  
    // Returns first token  
    char *token = strtok(str, "-"); 
  
    // Keep printing tokens while one of the 
    // delimiters present in str[]. 
    while (token != NULL) 
    { 
        printf("%s
", token); 
        token = strtok(NULL, "-"); 
    } 
  
    return 0; 
} 

Output:

Geeks
for
Geeks

Another Example of strtok() :

// C code to demonstrate working of 
// strtok 
#include <string.h> 
#include <stdio.h> 
  
// Driver function 
int main() 
{ 
 // Declaration of string 
    char gfg[100] = " Geeks - for - geeks - Contribute"; 
  
    // Declaration of delimiter 
    const char s[4] = "-"; 
    char* tok; 
  
    // Use of strtok 
    // get first token 
    tok = strtok(gfg, s); 
  
    // Checks for delimeter 
    while (tok != 0) { 
        printf(" %s
", tok); 
  
        // Use of strtok 
        // go through other tokens 
        tok = strtok(0, s); 
    } 
  
    return (0); 
} 

Output:

Geeks
 for
 geeks
 Contribute

strtok_r()

Just like strtok() function in C, strtok_r() does the same task of parsing a string into a sequence of tokens. strtok_r() is a reentrant version of strtok().

There are two ways we can call strtok_r()

// The third argument saveptr is a pointer to a char * 
// variable that is used internally by strtok_r() in 
// order to maintain context between successive calls
// that parse the same string.
char *strtok_r(char *str, const char *delim, char **saveptr);

Below is a simple C++ program to show the use of strtok_r() :

// C/C++ program to demonstrate working of strtok_r() 
// by splitting string based on space character. 
#include<stdio.h> 
#include<string.h> 
  
int main() 
{ 
    char str[] = "Geeks for Geeks"; 
    char *token; 
    char *rest = str; 
  
    while ((token = strtok_r(rest, " ", &rest))) 
        printf("%s
", token); 
  
    return(0); 
} 

Output:

Geeks
for
Geeks

Another Example of strtok() :

tags:

leave a comment

0 Comments

Subscribe to Our Newsletter

Top Followed books