added HBase timestamp support

[sbarnoud]

#210

What changes were proposed in this pull request?

Added support of HBase timestamp.

How was this patch tested?

New TU. Added control in TU that "reserved" column families (rowkey and timestamp) are not created.

[regata]

@sbarnoud when there are multiple cells in a single column, each cell will have different timestamp. However, this code reads the timestamp only from the latest cell. Is this intended behavior?

[regata]

plus tsSeq is not used inside buildRows

[regata]

how about this?

    val tsField = relation.catalog.getTimestamp.head // ADDED
    val valueSeq: Seq[Map[Long, (Field, Any)]] = fields.filter(c => HBaseTableCatalog.isNotReserved(c.cf)).map { x =>
      import scala.collection.JavaConverters.asScalaBufferConverter
      val dataType = SHCDataTypeFactory.create(x)
      val kvs = result.getColumnCells(
        relation.catalog.shcTableCoder.toBytes(x.cf),
        relation.catalog.shcTableCoder.toBytes(x.col)).asScala

      kvs.map(kv => {
        val v = CellUtil.cloneValue(kv)
        (kv.getTimestamp, x -> dataType.fromBytes(v))
      }).toMap.withDefaultValue(x -> null)
    }

    val ts = valueSeq.foldLeft(Set.empty[Long])((acc, map) => acc ++ map.keySet)
    //we are loosing duplicate here, because we didn't support passing version (timestamp) to the row
    ts.map(version => {
      val tsSeq = Seq((tsField, version)).toMap // ADDED
      keySeq ++ tsSeq ++ valueSeq.map(_.apply(version)).toMap // MODIFIED
    }).map { unioned =>
      // Return the row ordered by the requested order
      Row.fromSeq(fields.map(unioned.get(_).getOrElse(null)))
    }

The only extra change that is required is to specify timestamp column when mergeToLatest is false

[sbarnoud]

My bad.In my use case, i have a single version, and i forget this point.
Yes, we should have the ts corresponding to the version!