Save to file in JavaDStream

To my surprise, JavaDStream class (descendant of DStream?) has no save file method. Its, exactly as written, have DStream.

And, this approach works:

frames.dstream().saveAsTextFiles("hdfs://...

Launch spark streaming samples

Then you try to launch streaming sample, test RDD has no output data and, at first sight, no java error stacks, but include warning:

WARN BlockManager: Block input ... replicated to only 0 peer(s) instead of 1 peers

Possible reason is that spark can not execute submitted task, because can not create additional execution instances. It must be at least 2 for streaming. Setting more instances in submit command with same result.

spark-submit --class JavaCustomReceiver -master local[*]

Check user parameter

export SPARK_EXECUTOR_INSTANCES=N