diff options
| author | joyyoj <sunshch@gmail.com> | 2014-06-10 17:26:17 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-06-10 17:26:17 -0700 |
| commit | 29660443077619ee854025b8d0d3d64181724054 (patch) | |
| tree | 1976aec100ace16799a6f93fead00e5847eaf35a /external | |
| parent | 1abbde0e89131ad95e793ac1834c392db46b448e (diff) | |
| download | spark-29660443077619ee854025b8d0d3d64181724054.tar.gz spark-29660443077619ee854025b8d0d3d64181724054.tar.bz2 spark-29660443077619ee854025b8d0d3d64181724054.zip | |
[SPARK-1998] SparkFlumeEvent with body bigger than 1020 bytes are not re...
flume event sent to Spark will fail if the body is too large and numHeaders is greater than zero
Author: joyyoj <sunshch@gmail.com>
Closes #951 from joyyoj/master and squashes the following commits:
f4660c5 [joyyoj] [SPARK-1998] SparkFlumeEvent with body bigger than 1020 bytes are not read properly
Diffstat (limited to 'external')
| -rw-r--r-- | external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala b/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala index 5be33f1d5c..ed35e34ad4 100644 --- a/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala +++ b/external/flume/src/main/scala/org/apache/spark/streaming/flume/FlumeInputDStream.scala @@ -71,12 +71,12 @@ class SparkFlumeEvent() extends Externalizable { for (i <- 0 until numHeaders) { val keyLength = in.readInt() val keyBuff = new Array[Byte](keyLength) - in.read(keyBuff) + in.readFully(keyBuff) val key : String = Utils.deserialize(keyBuff) val valLength = in.readInt() val valBuff = new Array[Byte](valLength) - in.read(valBuff) + in.readFully(valBuff) val value : String = Utils.deserialize(valBuff) headers.put(key, value) |