Definition
- Stream.concat merges two streams into one.
- toMap(id, e, merge) deduplicates by a business key (employee id), not by object equality.
- Merge function encodes the dedup policy: keep first, keep latest, or merge fields.
- values().stream() extracts the deduplicated employees.
- groupingBy(dept, averagingDouble) computes per-department average salary.
At National Bank of India, Priya combines employee lists from two branches, removes anyone listed twice by their employee ID, then averages pay per department over the cleaned list.
Understanding the Concepts
What are the three steps?
- Merge: concatenate the two branch lists.
- Deduplicate: by employee ID using
toMapwith a merge function. - Aggregate: group by department and compute average salary.
How do you implement it?
package com.bank.hr;
import java.math.BigDecimal;
import java.util.*;
import java.util.stream.Collectors;
import java.util.stream.Stream;
public class MergeDedupeAggregate {
record Employee(String id, String name, String department, BigDecimal salary) {}
public static void main(String[] args) {
List<Employee> branchA = List.of(
new Employee("SAV2001", "Rajesh", "TRADING", new BigDecimal("1800000")),
new Employee("SAV2002", "Priya", "RISK", new BigDecimal("1200000")),
new Employee("SAV2003", "Meera", "TRADING", new BigDecimal("1500000"))
);
List<Employee> branchB = List.of(
new Employee("SAV2002", "Priya", "RISK", new BigDecimal("1200000")), // duplicate
new Employee("SAV2004", "Deepak", "RISK", new BigDecimal("900000")),
new Employee("SAV2005", "Kavita", "OPERATIONS", new BigDecimal("1100000"))
);
// Step 1: Merge
// Step 2: Deduplicate by id (keep first occurrence)
Collection<Employee> unique = Stream.concat(branchA.stream(), branchB.stream())
.collect(Collectors.toMap(
Employee::id,
e -> e,
(existing, replacement) -> existing)) // keep first
.values();
// Step 3: Group by department and average salary
Map<String, Double> avgByDept = unique.stream()
.collect(Collectors.groupingBy(
Employee::department,
Collectors.averagingDouble(e -> e.salary().doubleValue())));
System.out.println("Unique employees: " + unique.size());
avgByDept.forEach((dept, avg) ->
System.out.printf("%s: Rs.%.0f avg%n", dept, avg));
}
}
Unique employees: 5
TRADING: Rs.1650000 avg
RISK: Rs.1050000 avg
OPERATIONS: Rs.1100000 avg
Priya (SAV2002) appears in both lists but is counted only once.
Why use toMap for deduplication instead of distinct()?
distinct() uses equals() on the whole object. Here, uniqueness is defined by the employee ID, not by object equality. toMap keyed by ID with a merge function is the idiomatic way to deduplicate by a specific business key.
| Approach | Equality | Flexibility |
|---|---|---|
| distinct() | Object equals() | None: all-or-nothing |
| toMap(id, e, merge) | Business key (id) | Merge function controls policy |
What are common mistakes?
Mistake: Using distinct() when uniqueness is by a specific field.
// WRONG: distinct uses equals on the whole Employee
Stream.concat(branchA.stream(), branchB.stream())
.distinct() // only works if Employee.equals checks just id
.toList();
// CORRECT: toMap by id with merge function
Stream.concat(branchA.stream(), branchB.stream())
.collect(Collectors.toMap(Employee::id, e -> e, (a, b) -> a))
.values();
What merge policies are common?
// Keep first occurrence
(existing, replacement) -> existing
// Keep latest (last seen wins)
(existing, replacement) -> replacement
// Keep the one with higher salary
(existing, replacement) -> existing.salary().compareTo(replacement.salary()) >= 0
? existing : replacement
What about using BigDecimal for exact averages?
The averagingDouble collector uses double. For exact banking averages, use collectingAndThen with BigDecimal arithmetic as shown in Q2391.
What should a senior engineer know?
Distinct-by-a-key is best expressed as toMap keyed by that key with a merge function. The merge function also encodes the business policy. This merge-then-dedupe-then-aggregate pipeline is a very common data-integration shape.
What are common interview follow-up questions?
- "What if you want to keep the latest?" Change the merge to `(a, b) -> b`. - "What if duplicates have different salaries?" The merge function decides which to keep. - "Can you do it without toMap?" Use `filter` with a `Set<String>` tracking seen IDs, but `toMap` is cleaner.
Visual Overview
Branch A: [SAV2001/Rajesh, SAV2002/Priya, SAV2003/Meera]
Branch B: [SAV2002/Priya, SAV2004/Deepak, SAV2005/Kavita]
│
│ Stream.concat
▼
[Rajesh, Priya, Meera, Priya(dup), Deepak, Kavita]
│
│ toMap(id, e, keepFirst)
▼
{SAV2001=Rajesh, SAV2002=Priya, SAV2003=Meera,
SAV2004=Deepak, SAV2005=Kavita} ← Priya deduped
│
│ groupingBy(dept, averagingDouble)
▼
TRADING: avg(1.8M, 1.5M) = 1.65M
RISK: avg(1.2M, 0.9M) = 1.05M
OPS: avg(1.1M) = 1.1M
Key Takeaway
This is a three-step data-integration pipeline: merge, deduplicate by a business key, then aggregate. Stream.concat merges the two branch lists. toMap keyed by employee ID with a merge function deduplicates, keeping one entry per ID. The map's values() gives the clean set for groupingBy(dept, averagingDouble(salary)).
At the National Bank of India, this consolidates overlapping branch submissions into a clean per-department average. The key technique is distinct-by-a-key: since distinct() uses object equals(), and uniqueness here is defined by employee ID, toMap by ID with a merge function is the idiomatic approach.
The merge function encodes the business policy: keep first, keep latest, or keep the one with the higher salary. This flexibility is why toMap is preferred over distinct() for field-based deduplication.
Remember one sentence: concat to merge, toMap(id, e, merge) to deduplicate by a business key, then groupingBy to aggregate over the clean set.
---