diff options
author | Shivaram Venkataraman <shivaram@eecs.berkeley.edu> | 2013-08-06 15:43:46 -0700 |
---|---|---|
committer | Shivaram Venkataraman <shivaram@eecs.berkeley.edu> | 2013-08-06 15:43:46 -0700 |
commit | 471fbadd0c8cb8d310e3e1dd0e694e357ff1233e (patch) | |
tree | c8710554a0ce04e87873308540757eadfcbbd244 /core/src | |
parent | d2b0f0c23d9ccd5e8a23450e421503d3201f3450 (diff) | |
download | spark-471fbadd0c8cb8d310e3e1dd0e694e357ff1233e.tar.gz spark-471fbadd0c8cb8d310e3e1dd0e694e357ff1233e.tar.bz2 spark-471fbadd0c8cb8d310e3e1dd0e694e357ff1233e.zip |
Java examples, tests for KMeans and ALS
- Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it
easier to call from Java
- Renames class methods from `train` to `run` to enable static methods to be
called from Java.
- Add unit tests which check if both static / class methods can be called.
- Also add examples which port the main() function in ALS, KMeans to the
examples project.
Couple of minor changes to existing code:
- Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily
- Workaround a bug where using double[] from Java leads to class cast exception in
KMeans init
Diffstat (limited to 'core/src')
-rw-r--r-- | core/src/main/scala/spark/RDD.scala | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/core/src/main/scala/spark/RDD.scala b/core/src/main/scala/spark/RDD.scala index ca7cdd622a..160b3e9d83 100644 --- a/core/src/main/scala/spark/RDD.scala +++ b/core/src/main/scala/spark/RDD.scala @@ -31,6 +31,7 @@ import org.apache.hadoop.mapred.TextOutputFormat import it.unimi.dsi.fastutil.objects.{Object2LongOpenHashMap => OLMap} +import spark.api.java.JavaRDD import spark.broadcast.Broadcast import spark.Partitioner._ import spark.partial.BoundedDouble @@ -950,4 +951,8 @@ abstract class RDD[T: ClassManifest]( id, origin) + def toJavaRDD() : JavaRDD[T] = { + new JavaRDD(this)(elementClassManifest) + } + } |