Package org.apache.sysds.runtime.controlprogram.parfor

Class DataPartitionerRemoteSparkMapper

java.lang.Object

org.apache.sysds.runtime.controlprogram.parfor.ParWorker

org.apache.sysds.runtime.controlprogram.parfor.DataPartitionerRemoteSparkMapper

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.PairFlatMapFunction<scala.Tuple2<MatrixIndexes,MatrixBlock>,Long,org.apache.hadoop.io.Writable>

public class DataPartitionerRemoteSparkMapper
extends ParWorker
implements org.apache.spark.api.java.function.PairFlatMapFunction<scala.Tuple2<MatrixIndexes,MatrixBlock>,Long,org.apache.hadoop.io.Writable>

NOTE: for the moment we only support binary block here TODO extend impl for binarycell and textcell Interface of Writable output in order to support both PairWritableBlock and PairWritableCell.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
DataPartitionerRemoteSparkMapper(DataCharacteristics mc, Types.FileFormat fmt, ParForProgramBlock.PDataPartitionFormat dpf, int n)

Method Summary

Modifier and Type	Method	Description
Iterator<scala.Tuple2<Long,org.apache.hadoop.io.Writable>>	call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0)

Methods inherited from class org.apache.sysds.runtime.controlprogram.parfor.ParWorker

getExecutedIterations, getExecutedTasks, getExecutionContext, getVariables

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DataPartitionerRemoteSparkMapper

public DataPartitionerRemoteSparkMapper(DataCharacteristics mc,
 Types.FileFormat fmt,
 ParForProgramBlock.PDataPartitionFormat dpf,
 int n)

Method Details

call

public Iterator<scala.Tuple2<Long,org.apache.hadoop.io.Writable>> call(scala.Tuple2<MatrixIndexes,MatrixBlock> arg0)
                                                                throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.PairFlatMapFunction<scala.Tuple2<MatrixIndexes,MatrixBlock>,Long,org.apache.hadoop.io.Writable>

Throws:

Exception