All nodes/Data/DataFrame Ops

DF Distinct

SQL SELECT DISTINCT: removes duplicates over a set of columns.

DF Distinct▶

DataFrameSuccess

Error

Type in the graph: df_drop_duplicates

Exec

An error branch can be enabled (expose_error_output) to handle failures on their own path.

Ports can be split into separate execution and data handles.

Try it

Minimal working workflow

Entry

DF Distinct▶

Exit▶

Execute + Data

Press “Copy nodes”, open the editor and hit Ctrl+V on the canvas.

Runs as pasted

When to use it

Keeps one row per distinct set of values — the SQL SELECT DISTINCT of the family. It typically sits after several exports were stacked with df_concat, or before a mailing so one address does not get two letters. If sums or averages are needed as well, use df_groupby_agg: it forms the same groups but computes over them.

How it works

An empty “Columns” list means strict comparison: a duplicate is a row that matches another one across every column. List some columns and only those are compared, while the remaining values are taken from whichever row survived.

Which row survives is decided by “Keep occurrence”: first is the earliest one seen, last the latest. That is the order of rows in the incoming table, nothing else.

After the collapse, rows are renumbered from zero.

Common mistakes

“Keep the most recent” without sorting. first and last refer to position, not to a date. Put df_sort on the relevant column before this node, and only then does “last” mean “most recent”.
Expecting the surviving row to absorb values from the dropped ones. Extra rows are discarded whole; merging values is what df_groupby_agg does.
Assuming the comparison is clever. It is exact: Ivan, ivan and Ivan are three different values. Normalise the column first — for instance with df_update.
Naming a column the table does not have fails the node. Check the names against the previous step’s preview in the run log.

Inputs

Port	Wire	Payload	Notes
DataFrame`df`	Execute + Data`execute_data`	`dataframe`

Outputs

Port	Wire	Payload	Notes
Success`output`	Execute + Data`execute_data`	`dataframe`
Error`on_error`	Execute + Data`execute_data`	—	shown when expose_error_output = true

Configuration

Field Type Default Description

Field	Type	Default	Description
Columns`subset`	`array<string>`	—	Columns that define a duplicate (empty = every column must match).
Keep occurrence`keep`	`string`	`first`	Which row of a duplicate group survives. Options: `first`, `last`

Columnssubset

array<string>

—

Columns that define a duplicate (empty = every column must match).

Keep occurrencekeep

string

first

Which row of a duplicate group survives.

Options: first, last

Shared fields

Every node has these three — the platform adds them, not the node author.

expose_error_output — When enabled, show an execution output to connect nodes that run if this step fails.
split_ports_in — Show separate execution and data input handles instead of one combined port.
split_ports_out — Show separate execution and data output handles instead of one combined port.