pyspark.sql.streaming.DataStreamWriter.outputMode¶
-
DataStreamWriter.outputMode(outputMode)[source]¶ Specifies how data of a streaming DataFrame/Dataset is written to a streaming sink.
New in version 2.0.0.
Options include:
- append: Only the new rows in the streaming DataFrame/Dataset will be written to
the sink
- complete: All the rows in the streaming DataFrame/Dataset will be written to the sink
every time these are some updates
- update: only the rows that were updated in the streaming DataFrame/Dataset will be
written to the sink every time there are some updates. If the query doesn’t contain aggregations, it will be equivalent to append mode.
Notes
This API is evolving.
Examples
>>> writer = sdf.writeStream.outputMode('append')