Apache Beam: KvSwap

Overview

What if you have a Key-Value pair, but want to group on the values, not the keys? Should you write a custom doFn to switch the keys and values?

NO!

You should use the KvSwap transform!

When You Should Use the KvSwap Transform

When you want to easily swap the keys and values of a PCollection of KV pairs.

How to Use the KvSwap Transform

Just apply the built-in Transform to a PCollection of KVs. The output PCollection will have the Keys and Values swapped from the input KV.

Example: Swap Keys and Values from Word Counts

    // Create key/value pairs
    PCollection<KV<String, Integer>> pairs =
        pipeline.apply(
            Create.of(KV.of("one", 1), KV.of("two", 2), KV.of("three", 3), KV.of("four", 4)));
    // Returns KV collection with keys and values swapped: PCollection<KV<K,V>> ->
    // PCollection<KV<V,K>>
    PCollection<KV<Integer, String>> swap = pairs.apply(KvSwap.create());

Conclusion

Check out other useful transforms from the official Apache Beam documentation.

Apache Beam: KvSwap

Comments

Apache Beam and Google Cloud Dataflow

Apache Beam: Keys

More from this blog

Harnessing the IP Address  Data Type in Apache Druid (2025)

Geospatial Queries in Apache Druid: A Ride-Sharing Example

Automating Google Cloud with GitHub Actions using gcloud CLI

How to Run a Scheduled Bash Script with Google Cloud Platform

Google Dataflow Optimization: Streaming Engine

Overview

When You Should Use the KvSwap Transform

How to Use the KvSwap Transform

Example: Swap Keys and Values from Word Counts

Conclusion

Command Palette

Comments

Apache Beam and Google Cloud Dataflow

Apache Beam: Keys

More from this blog

Overview

When You Should Use the KvSwap Transform

How to Use the KvSwap Transform

Example: Swap Keys and Values from Word Counts

Conclusion