upper() method in Pyspark

The upper() method in Pyspark is used to convert a string column to uppercase.

Syntax pyspark.sql.functions.upper(“column1”)

In this exercise, we are using the datasource data.csv. You can download the datasource and use for the transformation.

Example: First create the SparkSession and read the data from the CSV file.

Python

# Import the SparkSession module
from pyspark.sql import SparkSession

# Initialize a Spark session
spark = SparkSession.builder.appName("App Name").getOrCreate()

# Import the Data
df = spark.read.csv("data.csv", header=True, inferSchema=True)

# Show the data in the DataFrame
df.show()

The output of the above code is shown below:

Python

# Import the functions module
from pyspark.sql.functions import *

df = df.withColumn('uppercase_name', upper(df['Name'])) 
df.show()

Converts the 'Name' column to uppercase. The output of the above code is shown below:

Previous Next