Apache Beam: Keys

Overview

What if you only care about the keys of your PCollection and not necessarily the values? Maybe you have KV pairs for words and their counts in a string (KV<String, Integer>) and want to use the list of words in another PCollection.

You should use the Keys transform!

When You Should Use the Keys Transform

When you only want the keys in from a KV PCollection.

How to Use the Keys Transform

Just apply the builtin Transform to a PCollection of KVs. The output PCollection will have the type of the Keys in the KV and be flattened for all input elements.

Example: Extract Keys from Word Counts

    // Create key/value pairs
    PCollection<KV<String, Integer>> pairs =
        pipeline.apply(
            Create.of(KV.of("one", 1), KV.of("two", 2), KV.of("three", 3), KV.of("four", 4)));
    // Returns only the values of the collection: PCollection<KV<K,V>> ->
    // PCollection<V>
    PCollection<String> valuesOnly = pairs.apply(Keys.create());

Conclusion

Check out other useful transforms from the official Apache Beam documentation.

Apache Beam: Keys

Comments

Apache Beam and Google Cloud Dataflow

Apache Beam: Filter

More from this blog

Harnessing the IP Address  Data Type in Apache Druid (2025)

Geospatial Queries in Apache Druid: A Ride-Sharing Example

Automating Google Cloud with GitHub Actions using gcloud CLI

How to Run a Scheduled Bash Script with Google Cloud Platform

Google Dataflow Optimization: Streaming Engine

Overview

When You Should Use the Keys Transform

How to Use the Keys Transform

Example: Extract Keys from Word Counts

Conclusion

Command Palette

Comments

Apache Beam and Google Cloud Dataflow

Apache Beam: Filter

More from this blog

Overview

When You Should Use the Keys Transform

How to Use the Keys Transform

Example: Extract Keys from Word Counts

Conclusion