Package org.apache.sysds.runtime.frame.data.columns

Class OptionalArray<T>

java.lang.Object
org.apache.sysds.runtime.frame.data.columns.Array<T>
org.apache.sysds.runtime.frame.data.columns.OptionalArray<T>
All Implemented Interfaces:
org.apache.hadoop.io.Writable
public class OptionalArray<T> extends Array<T>

  • Constructor Details

    • OptionalArray

      public OptionalArray(T[] a)

    • OptionalArray

      public OptionalArray(T[] a, Types.ValueType vt)

    • OptionalArray

      public OptionalArray(Array<T> a, boolean empty)

    • OptionalArray

      public OptionalArray(Array<T> a, ABooleanArray n)

  • Method Details

    • write

      public void write(DataOutput out) throws IOException
      Throws:
      IOException

    • getExactSerializedSize

      public long getExactSerializedSize()
      Description copied from class: Array
      Get the exact serialized size on disk of this array.
      Specified by:
      getExactSerializedSize in class Array<T>
      Returns:
      The exact size on disk

    • getInMemorySize

      public long getInMemorySize()
      Description copied from class: Array
      Get in memory size, not counting reference to this object.
      Overrides:
      getInMemorySize in class Array<T>
      Returns:
      the size in memory of this object.

    • readFields

      public void readFields(DataInput in) throws IOException
      Throws:
      IOException

    • get

      public T get(int index)
      Description copied from class: Array
      Get the value at a given index. This method returns objects that have a high overhead in allocation. Therefore it is not as efficient as using the vectorized operations specified in the object.
      Specified by:
      get in class Array<T>
      Parameters:
      index - The index to query
      Returns:
      The value returned as an object

    • getInternal

      public T getInternal(int index)
      Description copied from class: Array
      Get the internal value at a given index. For instance HashIntegerArray would return the underlying long not a string.
      Overrides:
      getInternal in class Array<T>
      Parameters:
      index - the index to get
      Returns:
      The value to get

    • get

      public Object get()
      Description copied from class: Array
      Get the underlying array out of the column Group, it is the responsibility of the caller to know what type it is. Also it is not guaranteed that the underlying data structure does not allocate an appropriate response to the caller. This in practice means that if called there is a possibility that the entire array is allocated again. So the method should only be used for debugging purposes not for performance.
      Specified by:
      get in class Array<T>
      Returns:
      The underlying array.

    • getAsDouble

      public double getAsDouble(int i)
      Description copied from class: Array
      Get the index's value. returns 0 in case of Null.
      Specified by:
      getAsDouble in class Array<T>
      Parameters:
      i - index to get value from
      Returns:
      the value

    • getAsNaNDouble

      public double getAsNaNDouble(int i)
      Description copied from class: Array
      Get the index's value. returns Double.NaN in case of Null.
      Overrides:
      getAsNaNDouble in class Array<T>
      Parameters:
      i - index to get value from
      Returns:
      the value

    • set

      public void set(int index, T value)
      Description copied from class: Array
      Set index to the given value of same type
      Specified by:
      set in class Array<T>
      Parameters:
      index - The index to set
      value - The value to assign

    • set

      public void set(int index, double value)
      Description copied from class: Array
      Set index to given double value (cast to the correct type of this array)
      Specified by:
      set in class Array<T>
      Parameters:
      index - the index to set
      value - the value to set it to (before casting to correct value type)

    • set

      public void set(int index, String value)
      Description copied from class: Array
      Set index to the given value of the string parsed.
      Specified by:
      set in class Array<T>
      Parameters:
      index - The index to set
      value - The value to assign

    • setFromOtherType

      public void setFromOtherType(int rl, int ru, Array<?> value)
      Description copied from class: Array
      Set range to given arrays value
      Specified by:
      setFromOtherType in class Array<T>
      Parameters:
      rl - row lower
      ru - row upper (inclusive)
      value - value array to take values from (other type)

    • set

      public void set(int rl, int ru, Array<T> value, int rlSrc)
      Description copied from class: Array
      Set range to given arrays value with an offset into other array
      Overrides:
      set in class Array<T>
      Parameters:
      rl - row lower
      ru - row upper (inclusive)
      value - value array to take values from
      rlSrc - the offset into the value array to take values from

    • setNz

      public void setNz(int rl, int ru, Array<T> value)
      Description copied from class: Array
      Set non default values in the range from the value array given
      Specified by:
      setNz in class Array<T>
      Parameters:
      rl - row start
      ru - row upper inclusive
      value - value array of same type

    • setFromOtherTypeNz

      public void setFromOtherTypeNz(int rl, int ru, Array<?> value)
      Description copied from class: Array
      Set non default values in the range from the value array given
      Specified by:
      setFromOtherTypeNz in class Array<T>
      Parameters:
      rl - row start
      ru - row end inclusive
      value - value array of different type

    • append

      public void append(String value)
      Description copied from class: Array
      Append a string value to the current Array, this should in general be avoided, and appending larger blocks at a time should be preferred.
      Specified by:
      append in class Array<T>
      Parameters:
      value - The value to append

    • append

      public void append(T value)
      Description copied from class: Array
      Append a value of the same type of the Array. This should in general be avoided, and appending larger blocks at a time should be preferred.
      Specified by:
      append in class Array<T>
      Parameters:
      value - The value to append

    • append

      public Array<T> append(Array<T> other)
      Description copied from class: Array
      Append other array, if the other array is fitting in current allocated size use that allocated size, otherwise allocate new array to combine the other with this. This method should use the set range function, and should be preferred over the append single values.
      Specified by:
      append in class Array<T>
      Parameters:
      other - The other array of same type to append to this.
      Returns:
      The combined arrays.

    • appendOther

      public static <T> OptionalArray<T> appendOther(OptionalArray<T> that, Array<T> appended)

    • slice

      public Array<T> slice(int rl, int ru)
      Description copied from class: Array
      Slice out the sub range and return new array with the specified type. If the conversion fails fallback to normal slice.
      Specified by:
      slice in class Array<T>
      Parameters:
      rl - row start
      ru - row end (not included)
      Returns:
      A new array of sub range.

    • reset

      public void reset(int size)
      Description copied from class: Array
      Reset the Array and set to a different size. This method is used to reuse an already allocated Array, without extra allocation. It should only be done in cases where the Array is no longer in use in any FrameBlocks.
      Specified by:
      reset in class Array<T>
      Parameters:
      size - The size to reallocate into.

    • getAsByteArray

      public byte[] getAsByteArray()
      Description copied from class: Array
      Return the current allocated Array as a byte[], this is used to serialize the allocated Arrays out to the PythonAPI.
      Specified by:
      getAsByteArray in class Array<T>
      Returns:
      The array as bytes

    • getValueType

      public Types.ValueType getValueType()
      Description copied from class: Array
      Get the current value type of this array.
      Specified by:
      getValueType in class Array<T>
      Returns:
      The current value type.

    • changeType

      public Array<?> changeType(Types.ValueType t)
      Description copied from class: Array
      Change the allocated array to a different type. If the type is the same a deep copy is returned for safety.
      Overrides:
      changeType in class Array<T>
      Parameters:
      t - The type to change to
      Returns:
      A new column array.

    • analyzeValueType

      public Pair<Types.ValueType,Boolean> analyzeValueType(int maxCells)
      Description copied from class: Array
      Analyze the column to figure out if the value type can be refined to a better type. The return is in two parts, first the type it can be, second if it contains nulls.
      Specified by:
      analyzeValueType in class Array<T>
      Parameters:
      maxCells - maximum number of cells to analyze
      Returns:
      A better or equivalent value type to represent the column, including null information.

    • getFrameArrayType

      public ArrayFactory.FrameArrayType getFrameArrayType()
      Description copied from class: Array
      Get the internal FrameArrayType, to specify the encoding of the Types, note there are more Frame Array Types than there is ValueTypes.
      Specified by:
      getFrameArrayType in class Array<T>
      Returns:
      The FrameArrayType

    • fill

      public void fill(String val)
      Description copied from class: Array
      fill the entire array with specific value.
      Specified by:
      fill in class Array<T>
      Parameters:
      val - the value to fill with.

    • fill

      public void fill(T val)
      Description copied from class: Array
      fill the entire array with specific value.
      Specified by:
      fill in class Array<T>
      Parameters:
      val - the value to fill with.

    • isShallowSerialize

      public boolean isShallowSerialize()
      Description copied from class: Array
      analyze if this array can be shallow serialized. to allow caching without modification.
      Specified by:
      isShallowSerialize in class Array<T>
      Returns:
      boolean saying true if shallow serialization is available

    • getNulls

      public ABooleanArray getNulls()
      Overrides:
      getNulls in class Array<T>

    • clone

      public Array<T> clone()
      Description copied from class: Array
      Overwrite of the java internal clone function for arrays, return a clone of underlying data that is mutable, (not immutable data.) Immutable data is dependent on the individual allocated arrays
      Specified by:
      clone in class Array<T>
      Returns:
      A clone

    • isEmpty

      public boolean isEmpty()
      Description copied from class: Array
      Get if this array is empty, aka filled with empty values.
      Specified by:
      isEmpty in class Array<T>
      Returns:
      boolean saying true if empty

    • select

      public Array<T> select(int[] indices)
      Description copied from class: Array
      Slice out the specified indices and return the sub array.
      Specified by:
      select in class Array<T>
      Parameters:
      indices - The indices to slice out
      Returns:
      the sliced out indices in an array format

    • select

      public Array<T> select(boolean[] select, int nTrue)
      Description copied from class: Array
      Slice out the true indices in the select input and return the sub array.
      Specified by:
      select in class Array<T>
      Parameters:
      select - a boolean vector specifying what to select
      nTrue - number of true values inside select
      Returns:
      the sliced out indices in an array format

    • isNotEmpty

      public final boolean isNotEmpty(int i)
      Specified by:
      isNotEmpty in class Array<T>

    • containsNull

      public boolean containsNull()
      Description copied from class: Array
      analyze if the array contains null values.
      Overrides:
      containsNull in class Array<T>
      Returns:
      If the array contains null.

    • hashDouble

      public double hashDouble(int idx)
      Description copied from class: Array
      Hash the given index of the array. It is allowed to return NaN on null elements.
      Specified by:
      hashDouble in class Array<T>
      Parameters:
      idx - The index to hash
      Returns:
      The hash value of that index.

    • equals

      public boolean equals(Array<T> other)
      Description copied from class: Array
      Equals operation on arrays.
      Specified by:
      equals in class Array<T>
      Parameters:
      other - The other array to compare to.
      Returns:
      True if the arrays are equivalent.

    • possiblyContainsNaN

      public boolean possiblyContainsNaN()
      Specified by:
      possiblyContainsNaN in class Array<T>

    • setM

      public void setM(HashMapToInt<T> map, AMapToData m, int i)
      Description copied from class: Array
      Set the index i in the map given based on the mapping provided. The map should be guaranteed to contain all unique values.
      Overrides:
      setM in class Array<T>
      Parameters:
      map - A map containing all unique values of this array
      m - The MapToData to set the value part of the Map from
      i - The index to set in m

    • setM

      public void setM(HashMapToInt<T> map, int si, AMapToData m, int i)
      Description copied from class: Array
      Set the index i in the map given based on the mapping provided. The map should be guaranteed to contain all unique values except null. Therefore in case of null we set the provided si value.
      Overrides:
      setM in class Array<T>
      Parameters:
      map - A map containing all unique values of this array
      si - The default value to use in m if this Array contains null at index i
      m - The MapToData to set the value part of the Map from
      i - The index to set in m

    • toString

      public String toString()
      Overrides:
      toString in class Array<T>