# Pivot and Unpivot ## Input Data ```python df = pd.DataFrame({'team': ['A', 'B', 'C', 'D', 'E'], 'points': [18, 22, 19, 14, 14], 'assists': [5, 7, 7, 9, 12], 'rebounds': [11, 8, 10, 6, 6]}) ``` ||team|points|assists|rebounds| |---|---|---|---|---| |0|A|18|5|11| |1|B|22|7|8| |2|C|19|7|10| |3|D|14|9|6| |4|E|14|12|6| ## R ## Python ### Using `stack` Method ```python df_melted = df.set_index("team").stack().reset_index() df_melted.columns = ["team", "type", "value"] df_melted ``` ||team|type|value| |---|---|---|---| |0|A|points|18| |1|A|assists|5| |2|A|rebounds|11| |3|B|points|22| |4|B|assists|7| |5|B|rebounds|8| |6|C|points|19| |7|C|assists|7| |8|C|rebounds|10| |9|D|points|14| |10|D|assists|9| |11|D|rebounds|6| |12|E|points|14| |13|E|assists|12| |14|E|rebounds|6| ### Using `pd.DataFrame.melt` Method ```python df.melt(id_vars="team", var_name="type", value_name="value") ``` ||team|type|value| |---|---|---|---| |0|A|points|18| |1|B|points|22| |2|C|points|19| |3|D|points|14| |4|E|points|14| |5|A|assists|5| |6|B|assists|7| |7|C|assists|7| |8|D|assists|9| |9|E|assists|12| |10|A|rebounds|11| |11|B|rebounds|8| |12|C|rebounds|10| |13|D|rebounds|6| |14|E|rebounds|6| ## Polars ## PySpark ```python from pyspark.sql import SparkSession from pyspark.sql.functions import explode, array, struct, lit spark = SparkSession.builder \ .appName("My App") \ .getOrCreate() # Convert Pandas DataFrame to Spark DataFrame spark_df = spark.createDataFrame(df) # Create an array of structs with column names and values melted_df = spark_df.withColumn( "values", explode( array( struct(lit("points").alias("type"), col("points").alias("value")), struct(lit("assists").alias("type"), col("assists").alias("value")), struct(lit("rebounds").alias("type"), col("rebounds").alias("value")) ) ) ).select( col("team"), col("values.type"), col("values.value") ) # Show the result melted_df.show() ``` ``` +----+--------+-----+ |team| type|value| +----+--------+-----+ | A| points| 18| | A| assists| 5| | A|rebounds| 11| | B| points| 22| | B| assists| 7| | B|rebounds| 8| | C| points| 19| | C| assists| 7| | C|rebounds| 10| | D| points| 14| | D| assists| 9| | D|rebounds| 6| | E| points| 14| | E| assists| 12| | E|rebounds| 6| +----+--------+-----+ ``` Note ```python array( struct(lit("points").alias("type"), col("points").alias("value")), struct(lit("assists").alias("type"), col("assists").alias("value")), struct(lit("rebounds").alias("type"), col("rebounds").alias("value")) ) ``` Here's an illustrative example of what one row's output might look like if you have a row in your DataFrame with values like `team: A, points: 18, assists: 5, rebounds: 11`: ``` ``` ![[IMG_3162.jpeg]] ![[IMG_3163.jpeg]]