diff options
| author | Yury Liavitski <seconds.before@gmail.com> | 2016-03-07 10:54:33 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-03-07 10:54:33 +0000 |
| commit | 03f57a6c2dd6ffd4038ca9cecbfc221deaf52393 (patch) | |
| tree | 5a886c71376a0ec8ebc8678e173af0a9cbd71f45 /examples/src | |
| parent | 4b13896ebf7cecf9d50514a62165b612ee18124a (diff) | |
| download | spark-03f57a6c2dd6ffd4038ca9cecbfc221deaf52393.tar.gz spark-03f57a6c2dd6ffd4038ca9cecbfc221deaf52393.tar.bz2 spark-03f57a6c2dd6ffd4038ca9cecbfc221deaf52393.zip | |
Fixing the type of the sentiment happiness value
## What changes were proposed in this pull request?
Added the conversion to int for the 'happiness value' read from the file. Otherwise, later on line 75 the multiplication will multiply a string by a number, yielding values like "-2-2" instead of -4.
## How was this patch tested?
Tested manually.
Author: Yury Liavitski <seconds.before@gmail.com>
Author: Yury Liavitski <yury.liavitski@il111.ice.local>
Closes #11540 from heliocentrist/fix-sentiment-value-type.
Diffstat (limited to 'examples/src')
| -rw-r--r-- | examples/src/main/scala/org/apache/spark/examples/streaming/TwitterHashTagJoinSentiments.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterHashTagJoinSentiments.scala b/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterHashTagJoinSentiments.scala index edf0e0b7b2..a8d392ca35 100644 --- a/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterHashTagJoinSentiments.scala +++ b/examples/src/main/scala/org/apache/spark/examples/streaming/TwitterHashTagJoinSentiments.scala @@ -56,8 +56,8 @@ object TwitterHashTagJoinSentiments { val wordSentimentFilePath = "data/streaming/AFINN-111.txt" val wordSentiments = ssc.sparkContext.textFile(wordSentimentFilePath).map { line => val Array(word, happinessValue) = line.split("\t") - (word, happinessValue) - } cache() + (word, happinessValue.toInt) + }.cache() // Determine the hash tags with the highest sentiment values by joining the streaming RDD // with the static RDD inside the transform() method and then multiplying |