diff options
| author | Joseph K. Bradley <joseph@databricks.com> | 2015-07-16 22:26:59 -0700 |
|---|---|---|
| committer | Xiangrui Meng <meng@databricks.com> | 2015-07-16 22:26:59 -0700 |
| commit | 322d286bb7773389ed07df96290e427b21c775bd (patch) | |
| tree | 953ab65861c3b646dc6097c198b5d085d0ee881e /sql | |
| parent | f893955b9cc6ea456fc5845890893c08d8878481 (diff) | |
| download | spark-322d286bb7773389ed07df96290e427b21c775bd.tar.gz spark-322d286bb7773389ed07df96290e427b21c775bd.tar.bz2 spark-322d286bb7773389ed07df96290e427b21c775bd.zip | |
[SPARK-7131] [ML] Copy Decision Tree, Random Forest impl to spark.ml
This PR copies the RandomForest implementation from spark.mllib to spark.ml. Note that this includes the DecisionTree implementation, but not the GradientBoostedTrees one (which will come later).
I essentially copied a minimal amount of code to spark.ml, removed the use of bins (and only used splits), and modified code only as much as necessary to get it to compile. The spark.ml implementation still uses some spark.mllib classes (privately), which can be moved in future PRs.
This refactoring will be helpful in extending the node representation to include more information, such as class probabilities.
Specifically:
* Copied code from spark.mllib to spark.ml:
* mllib.tree.DecisionTree, mllib.tree.RandomForest copied to ml.tree.impl.RandomForest (main implementation)
* NodeIdCache (needed to use splits instead of bins)
* TreePoint (use splits instead of bins)
* Added ml.tree.LearningNode used in RandomForest training (needed vars)
* Removed bins from implementation, and only used splits
* Small fix in JavaDecisionTreeRegressorSuite
CC: mengxr manishamde codedeft chouqin
Author: Joseph K. Bradley <joseph@databricks.com>
Closes #7294 from jkbradley/dt-move-impl and squashes the following commits:
48749be [Joseph K. Bradley] cleanups based on code review, mostly style
bea9703 [Joseph K. Bradley] scala style fixes. added some scala doc
4e6d2a4 [Joseph K. Bradley] removed unnecessary use of copyValues, setParent for trees
9a4d721 [Joseph K. Bradley] cleanups. removed InfoGainStats from ml, using old one for now.
836e7d4 [Joseph K. Bradley] Fixed test suite failures
bd5e063 [Joseph K. Bradley] fixed bucketizing issue
0df3759 [Joseph K. Bradley] Need to remove use of Bucketizer
d5224a9 [Joseph K. Bradley] modified tree and forest to use moved impl
cc01823 [Joseph K. Bradley] still editing RF to get it to work
19143fb [Joseph K. Bradley] More progress, but not done yet. Rebased with master after 1.4 release.
Diffstat (limited to 'sql')
0 files changed, 0 insertions, 0 deletions