diff options
| author | Reynold Xin <rxin@databricks.com> | 2016-12-04 21:51:10 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-12-04 21:51:10 -0800 |
| commit | e9730b707ddf6e344de3b3b8f43487f7b0f18e25 (patch) | |
| tree | 4cd4e931c62488e87d9100970e9c5a5e6bd45c56 /mllib | |
| parent | b019b3a8ac49336e657f5e093fa2fba77f8d12d2 (diff) | |
| download | spark-e9730b707ddf6e344de3b3b8f43487f7b0f18e25.tar.gz spark-e9730b707ddf6e344de3b3b8f43487f7b0f18e25.tar.bz2 spark-e9730b707ddf6e344de3b3b8f43487f7b0f18e25.zip | |
[SPARK-18702][SQL] input_file_block_start and input_file_block_length
## What changes were proposed in this pull request?
We currently have function input_file_name to get the path of the input file, but don't have functions to get the block start offset and length. This patch introduces two functions:
1. input_file_block_start: returns the file block start offset, or -1 if not available.
2. input_file_block_length: returns the file block length, or -1 if not available.
## How was this patch tested?
Updated existing test cases in ColumnExpressionSuite that covered input_file_name to also cover the two new functions.
Author: Reynold Xin <rxin@databricks.com>
Closes #16133 from rxin/SPARK-18702.
Diffstat (limited to 'mllib')
0 files changed, 0 insertions, 0 deletions