lower() method in PySpark

The lower() method converts a string column to lowercase.

Syntax pyspark.sql.functions.lower(“column1”)

In this exercise, we are using the datasource data.csv. You can download the datasource and use for the transformation.

Example: First create the SparkSession and read the data from the CSV file.

Python

# Import the SparkSession module
from pyspark.sql import SparkSession

# Initialize a Spark session
spark = SparkSession.builder.appName("App Name").getOrCreate()

# Import the Data
df = spark.read.csv("data.csv", header=True, inferSchema=True)

# Show the data in the DataFrame
df.show()

The output of the above code is shown below:

Let’s convert the 'Name' column to lowercase.

Python

# Import the functions module
from pyspark.sql.functions import *

df = df.withColumn('lowercase_name', lower(df['Name']))
df.show()

The output of the above code is shown below:

Previous Next