<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>pyspark.ml package — PySpark 1.3.1 documentation</title>
<link rel="stylesheet" href="_static/nature.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '1.3.1',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="top" title="PySpark 1.3.1 documentation" href="index.html" />
<link rel="next" title="pyspark.mllib package" href="pyspark.mllib.html" />
<link rel="prev" title="pyspark.streaming module" href="pyspark.streaming.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="pyspark.mllib.html" title="pyspark.mllib package"
accesskey="N">next</a></li>
<li class="right" >
<a href="pyspark.streaming.html" title="pyspark.streaming module"
accesskey="P">previous</a> |</li>
<li><a href="index.html">PySpark 1.3.1 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="pyspark-ml-package">
<h1>pyspark.ml package<a class="headerlink" href="#pyspark-ml-package" title="Permalink to this headline">¶</a></h1>
<div class="section" id="module-pyspark.ml">
<span id="module-context"></span><h2>Module Context<a class="headerlink" href="#module-pyspark.ml" title="Permalink to this headline">¶</a></h2>
<dl class="class">
<dt id="pyspark.ml.Param">
<em class="property">class </em><tt class="descclassname">pyspark.ml.</tt><tt class="descname">Param</tt><big>(</big><em>parent</em>, <em>name</em>, <em>doc</em>, <em>defaultValue=None</em><big>)</big><a class="headerlink" href="#pyspark.ml.Param" title="Permalink to this definition">¶</a></dt>
<dd><p>A param with self-contained documentation and optionally default value.</p>
</dd></dl>
<dl class="class">
<dt id="pyspark.ml.Params">
<em class="property">class </em><tt class="descclassname">pyspark.ml.</tt><tt class="descname">Params</tt><a class="headerlink" href="#pyspark.ml.Params" title="Permalink to this definition">¶</a></dt>
<dd><p>Components that take parameters. This also provides an internal
param map to store parameter values attached to the instance.</p>
<dl class="attribute">
<dt id="pyspark.ml.Params.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.Params.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<a class="reference internal" href="#pyspark.ml.Param" title="pyspark.ml.Param"><tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt></a>.</p>
</dd></dl>
</dd></dl>
<dl class="class">
<dt id="pyspark.ml.Transformer">
<em class="property">class </em><tt class="descclassname">pyspark.ml.</tt><tt class="descname">Transformer</tt><a class="headerlink" href="#pyspark.ml.Transformer" title="Permalink to this definition">¶</a></dt>
<dd><p>Abstract class for transformers that transform one dataset into
another.</p>
<dl class="attribute">
<dt id="pyspark.ml.Transformer.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.Transformer.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<a class="reference internal" href="#pyspark.ml.Param" title="pyspark.ml.Param"><tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.Transformer.transform">
<tt class="descname">transform</tt><big>(</big><em>dataset</em>, <em>params={}</em><big>)</big><a class="headerlink" href="#pyspark.ml.Transformer.transform" title="Permalink to this definition">¶</a></dt>
<dd><p>Transforms the input dataset with optional parameters.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>dataset</strong> – input dataset, which is an instance of
<a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><tt class="xref py py-class docutils literal"><span class="pre">pyspark.sql.DataFrame</span></tt></a></li>
<li><strong>params</strong> – an optional param map that overwrites embedded
params</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">transformed dataset</p>
</td>
</tr>
</tbody>
</table>
</dd></dl>
</dd></dl>
<dl class="class">
<dt id="pyspark.ml.Estimator">
<em class="property">class </em><tt class="descclassname">pyspark.ml.</tt><tt class="descname">Estimator</tt><a class="headerlink" href="#pyspark.ml.Estimator" title="Permalink to this definition">¶</a></dt>
<dd><p>Abstract class for estimators that fit models to data.</p>
<dl class="method">
<dt id="pyspark.ml.Estimator.fit">
<tt class="descname">fit</tt><big>(</big><em>dataset</em>, <em>params={}</em><big>)</big><a class="headerlink" href="#pyspark.ml.Estimator.fit" title="Permalink to this definition">¶</a></dt>
<dd><p>Fits a model to the input dataset with optional parameters.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>dataset</strong> – input dataset, which is an instance of
<a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><tt class="xref py py-class docutils literal"><span class="pre">pyspark.sql.DataFrame</span></tt></a></li>
<li><strong>params</strong> – an optional param map that overwrites embedded
params</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">fitted model</p>
</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.Estimator.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.Estimator.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<a class="reference internal" href="#pyspark.ml.Param" title="pyspark.ml.Param"><tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt></a>.</p>
</dd></dl>
</dd></dl>
<dl class="class">
<dt id="pyspark.ml.Pipeline">
<em class="property">class </em><tt class="descclassname">pyspark.ml.</tt><tt class="descname">Pipeline</tt><big>(</big><em>*args</em>, <em>**kwargs</em><big>)</big><a class="headerlink" href="#pyspark.ml.Pipeline" title="Permalink to this definition">¶</a></dt>
<dd><p>A simple pipeline, which acts as an estimator. A Pipeline consists
of a sequence of stages, each of which is either an
<a class="reference internal" href="#pyspark.ml.Estimator" title="pyspark.ml.Estimator"><tt class="xref py py-class docutils literal"><span class="pre">Estimator</span></tt></a> or a <a class="reference internal" href="#pyspark.ml.Transformer" title="pyspark.ml.Transformer"><tt class="xref py py-class docutils literal"><span class="pre">Transformer</span></tt></a>. When
<a class="reference internal" href="#pyspark.ml.Pipeline.fit" title="pyspark.ml.Pipeline.fit"><tt class="xref py py-meth docutils literal"><span class="pre">Pipeline.fit()</span></tt></a> is called, the stages are executed in
order. If a stage is an <a class="reference internal" href="#pyspark.ml.Estimator" title="pyspark.ml.Estimator"><tt class="xref py py-class docutils literal"><span class="pre">Estimator</span></tt></a>, its
<a class="reference internal" href="#pyspark.ml.Estimator.fit" title="pyspark.ml.Estimator.fit"><tt class="xref py py-meth docutils literal"><span class="pre">Estimator.fit()</span></tt></a> method will be called on the input
dataset to fit a model. Then the model, which is a transformer,
will be used to transform the dataset as the input to the next
stage. If a stage is a <a class="reference internal" href="#pyspark.ml.Transformer" title="pyspark.ml.Transformer"><tt class="xref py py-class docutils literal"><span class="pre">Transformer</span></tt></a>, its
<a class="reference internal" href="#pyspark.ml.Transformer.transform" title="pyspark.ml.Transformer.transform"><tt class="xref py py-meth docutils literal"><span class="pre">Transformer.transform()</span></tt></a> method will be called to produce
the dataset for the next stage. The fitted model from a
<a class="reference internal" href="#pyspark.ml.Pipeline" title="pyspark.ml.Pipeline"><tt class="xref py py-class docutils literal"><span class="pre">Pipeline</span></tt></a> is an <tt class="xref py py-class docutils literal"><span class="pre">PipelineModel</span></tt>, which
consists of fitted models and transformers, corresponding to the
pipeline stages. If there are no stages, the pipeline acts as an
identity transformer.</p>
<dl class="method">
<dt id="pyspark.ml.Pipeline.fit">
<tt class="descname">fit</tt><big>(</big><em>dataset</em>, <em>params={}</em><big>)</big><a class="headerlink" href="#pyspark.ml.Pipeline.fit" title="Permalink to this definition">¶</a></dt>
<dd><p>Fits a model to the input dataset with optional parameters.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>dataset</strong> – input dataset, which is an instance of
<a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><tt class="xref py py-class docutils literal"><span class="pre">pyspark.sql.DataFrame</span></tt></a></li>
<li><strong>params</strong> – an optional param map that overwrites embedded
params</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">fitted model</p>
</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.Pipeline.getStages">
<tt class="descname">getStages</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.Pipeline.getStages" title="Permalink to this definition">¶</a></dt>
<dd><p>Get pipeline stages.</p>
</dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.Pipeline.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.Pipeline.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<a class="reference internal" href="#pyspark.ml.Param" title="pyspark.ml.Param"><tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.Pipeline.setParams">
<tt class="descname">setParams</tt><big>(</big><em>self</em>, <em>stages=</em><span class="optional">[</span><span class="optional">]</span><big>)</big><a class="headerlink" href="#pyspark.ml.Pipeline.setParams" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets params for Pipeline.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.Pipeline.setStages">
<tt class="descname">setStages</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.Pipeline.setStages" title="Permalink to this definition">¶</a></dt>
<dd><p>Set pipeline stages.
:param value: a list of transformers or estimators
:return: the pipeline instance</p>
</dd></dl>
</dd></dl>
</div>
<div class="section" id="module-pyspark.ml.feature">
<span id="pyspark-ml-feature-module"></span><h2>pyspark.ml.feature module<a class="headerlink" href="#module-pyspark.ml.feature" title="Permalink to this headline">¶</a></h2>
<dl class="class">
<dt id="pyspark.ml.feature.Tokenizer">
<em class="property">class </em><tt class="descclassname">pyspark.ml.feature.</tt><tt class="descname">Tokenizer</tt><big>(</big><em>*args</em>, <em>**kwargs</em><big>)</big><a class="reference internal" href="_modules/pyspark/ml/feature.html#Tokenizer"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Tokenizer" title="Permalink to this definition">¶</a></dt>
<dd><p>A tokenizer that converts the input string to lowercase and then
splits it by white spaces.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.sql</span> <span class="kn">import</span> <span class="n">Row</span>
<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">([</span><span class="n">Row</span><span class="p">(</span><span class="n">text</span><span class="o">=</span><span class="s">"a b c"</span><span class="p">)])</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">tokenizer</span> <span class="o">=</span> <span class="n">Tokenizer</span><span class="p">(</span><span class="n">inputCol</span><span class="o">=</span><span class="s">"text"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s">"words"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">tokenizer</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
<span class="go">Row(text=u'a b c', words=[u'a', u'b', u'c'])</span>
<span class="gp">>>> </span><span class="c"># Change a parameter.</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">tokenizer</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="n">outputCol</span><span class="o">=</span><span class="s">"tokens"</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
<span class="go">Row(text=u'a b c', tokens=[u'a', u'b', u'c'])</span>
<span class="gp">>>> </span><span class="c"># Temporarily modify a parameter.</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">tokenizer</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="p">{</span><span class="n">tokenizer</span><span class="o">.</span><span class="n">outputCol</span><span class="p">:</span> <span class="s">"words"</span><span class="p">})</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
<span class="go">Row(text=u'a b c', words=[u'a', u'b', u'c'])</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">tokenizer</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span>
<span class="go">Row(text=u'a b c', tokens=[u'a', u'b', u'c'])</span>
<span class="gp">>>> </span><span class="c"># Must use keyword arguments to specify params.</span>
<span class="gp">>>> </span><span class="n">tokenizer</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="s">"text"</span><span class="p">)</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="o">...</span>
<span class="gr">TypeError</span>: <span class="n">Method setParams forces keyword arguments.</span>
</pre></div>
</div>
<dl class="method">
<dt id="pyspark.ml.feature.Tokenizer.getInputCol">
<tt class="descname">getInputCol</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.getInputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of inputCol or its default value.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.Tokenizer.getOutputCol">
<tt class="descname">getOutputCol</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.getOutputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of outputCol or its default value.</p>
</dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.feature.Tokenizer.inputCol">
<tt class="descname">inputCol</tt><em class="property"> = Param(parent=undefined, name='inputCol', doc='input column name', defaultValue='input')</em><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.inputCol" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.feature.Tokenizer.outputCol">
<tt class="descname">outputCol</tt><em class="property"> = Param(parent=undefined, name='outputCol', doc='output column name', defaultValue='output')</em><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.outputCol" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.feature.Tokenizer.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.Tokenizer.setInputCol">
<tt class="descname">setInputCol</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.setInputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Tokenizer.inputCol" title="pyspark.ml.feature.Tokenizer.inputCol"><tt class="xref py py-attr docutils literal"><span class="pre">inputCol</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.Tokenizer.setOutputCol">
<tt class="descname">setOutputCol</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.setOutputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.Tokenizer.outputCol" title="pyspark.ml.feature.Tokenizer.outputCol"><tt class="xref py py-attr docutils literal"><span class="pre">outputCol</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.Tokenizer.setParams">
<tt class="descname">setParams</tt><big>(</big><em>self</em>, <em>inputCol="input"</em>, <em>outputCol="output"</em><big>)</big><a class="reference internal" href="_modules/pyspark/ml/feature.html#Tokenizer.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.setParams" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets params for this Tokenizer.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.Tokenizer.transform">
<tt class="descname">transform</tt><big>(</big><em>dataset</em>, <em>params={}</em><big>)</big><a class="headerlink" href="#pyspark.ml.feature.Tokenizer.transform" title="Permalink to this definition">¶</a></dt>
<dd><p>Transforms the input dataset with optional parameters.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>dataset</strong> – input dataset, which is an instance of
<a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><tt class="xref py py-class docutils literal"><span class="pre">pyspark.sql.DataFrame</span></tt></a></li>
<li><strong>params</strong> – an optional param map that overwrites embedded
params</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">transformed dataset</p>
</td>
</tr>
</tbody>
</table>
</dd></dl>
</dd></dl>
<dl class="class">
<dt id="pyspark.ml.feature.HashingTF">
<em class="property">class </em><tt class="descclassname">pyspark.ml.feature.</tt><tt class="descname">HashingTF</tt><big>(</big><em>*args</em>, <em>**kwargs</em><big>)</big><a class="reference internal" href="_modules/pyspark/ml/feature.html#HashingTF"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.HashingTF" title="Permalink to this definition">¶</a></dt>
<dd><p>Maps a sequence of terms to their term frequencies using the
hashing trick.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.sql</span> <span class="kn">import</span> <span class="n">Row</span>
<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">([</span><span class="n">Row</span><span class="p">(</span><span class="n">words</span><span class="o">=</span><span class="p">[</span><span class="s">"a"</span><span class="p">,</span> <span class="s">"b"</span><span class="p">,</span> <span class="s">"c"</span><span class="p">])])</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">hashingTF</span> <span class="o">=</span> <span class="n">HashingTF</span><span class="p">(</span><span class="n">numFeatures</span><span class="o">=</span><span class="mi">10</span><span class="p">,</span> <span class="n">inputCol</span><span class="o">=</span><span class="s">"words"</span><span class="p">,</span> <span class="n">outputCol</span><span class="o">=</span><span class="s">"features"</span><span class="p">)</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hashingTF</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">features</span>
<span class="go">(10,[7,8,9],[1.0,1.0,1.0])</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hashingTF</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="n">outputCol</span><span class="o">=</span><span class="s">"freqs"</span><span class="p">)</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">freqs</span>
<span class="go">(10,[7,8,9],[1.0,1.0,1.0])</span>
<span class="gp">>>> </span><span class="n">params</span> <span class="o">=</span> <span class="p">{</span><span class="n">hashingTF</span><span class="o">.</span><span class="n">numFeatures</span><span class="p">:</span> <span class="mi">5</span><span class="p">,</span> <span class="n">hashingTF</span><span class="o">.</span><span class="n">outputCol</span><span class="p">:</span> <span class="s">"vector"</span><span class="p">}</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">hashingTF</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">params</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">vector</span>
<span class="go">(5,[2,3,4],[1.0,1.0,1.0])</span>
</pre></div>
</div>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.getInputCol">
<tt class="descname">getInputCol</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getInputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of inputCol or its default value.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.getNumFeatures">
<tt class="descname">getNumFeatures</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getNumFeatures" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of numFeatures or its default value.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.getOutputCol">
<tt class="descname">getOutputCol</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.feature.HashingTF.getOutputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of outputCol or its default value.</p>
</dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.feature.HashingTF.inputCol">
<tt class="descname">inputCol</tt><em class="property"> = Param(parent=undefined, name='inputCol', doc='input column name', defaultValue='input')</em><a class="headerlink" href="#pyspark.ml.feature.HashingTF.inputCol" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.feature.HashingTF.numFeatures">
<tt class="descname">numFeatures</tt><em class="property"> = Param(parent=undefined, name='numFeatures', doc='number of features', defaultValue=262144)</em><a class="headerlink" href="#pyspark.ml.feature.HashingTF.numFeatures" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.feature.HashingTF.outputCol">
<tt class="descname">outputCol</tt><em class="property"> = Param(parent=undefined, name='outputCol', doc='output column name', defaultValue='output')</em><a class="headerlink" href="#pyspark.ml.feature.HashingTF.outputCol" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.feature.HashingTF.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.feature.HashingTF.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.setInputCol">
<tt class="descname">setInputCol</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setInputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.HashingTF.inputCol" title="pyspark.ml.feature.HashingTF.inputCol"><tt class="xref py py-attr docutils literal"><span class="pre">inputCol</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.setNumFeatures">
<tt class="descname">setNumFeatures</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setNumFeatures" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.HashingTF.numFeatures" title="pyspark.ml.feature.HashingTF.numFeatures"><tt class="xref py py-attr docutils literal"><span class="pre">numFeatures</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.setOutputCol">
<tt class="descname">setOutputCol</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setOutputCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.feature.HashingTF.outputCol" title="pyspark.ml.feature.HashingTF.outputCol"><tt class="xref py py-attr docutils literal"><span class="pre">outputCol</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.setParams">
<tt class="descname">setParams</tt><big>(</big><em>self</em>, <em>numFeatures=1 << 18</em>, <em>inputCol="input"</em>, <em>outputCol="output"</em><big>)</big><a class="reference internal" href="_modules/pyspark/ml/feature.html#HashingTF.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.feature.HashingTF.setParams" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets params for this HashingTF.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.feature.HashingTF.transform">
<tt class="descname">transform</tt><big>(</big><em>dataset</em>, <em>params={}</em><big>)</big><a class="headerlink" href="#pyspark.ml.feature.HashingTF.transform" title="Permalink to this definition">¶</a></dt>
<dd><p>Transforms the input dataset with optional parameters.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>dataset</strong> – input dataset, which is an instance of
<a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><tt class="xref py py-class docutils literal"><span class="pre">pyspark.sql.DataFrame</span></tt></a></li>
<li><strong>params</strong> – an optional param map that overwrites embedded
params</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">transformed dataset</p>
</td>
</tr>
</tbody>
</table>
</dd></dl>
</dd></dl>
</div>
<div class="section" id="module-pyspark.ml.classification">
<span id="pyspark-ml-classification-module"></span><h2>pyspark.ml.classification module<a class="headerlink" href="#module-pyspark.ml.classification" title="Permalink to this headline">¶</a></h2>
<dl class="class">
<dt id="pyspark.ml.classification.LogisticRegression">
<em class="property">class </em><tt class="descclassname">pyspark.ml.classification.</tt><tt class="descname">LogisticRegression</tt><big>(</big><em>*args</em>, <em>**kwargs</em><big>)</big><a class="reference internal" href="_modules/pyspark/ml/classification.html#LogisticRegression"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression" title="Permalink to this definition">¶</a></dt>
<dd><p>Logistic regression.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.sql</span> <span class="kn">import</span> <span class="n">Row</span>
<span class="gp">>>> </span><span class="kn">from</span> <span class="nn">pyspark.mllib.linalg</span> <span class="kn">import</span> <span class="n">Vectors</span>
<span class="gp">>>> </span><span class="n">df</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">([</span>
<span class="gp">... </span> <span class="n">Row</span><span class="p">(</span><span class="n">label</span><span class="o">=</span><span class="mf">1.0</span><span class="p">,</span> <span class="n">features</span><span class="o">=</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">(</span><span class="mf">1.0</span><span class="p">)),</span>
<span class="gp">... </span> <span class="n">Row</span><span class="p">(</span><span class="n">label</span><span class="o">=</span><span class="mf">0.0</span><span class="p">,</span> <span class="n">features</span><span class="o">=</span><span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">[],</span> <span class="p">[]))])</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
<span class="gp">>>> </span><span class="n">lr</span> <span class="o">=</span> <span class="n">LogisticRegression</span><span class="p">(</span><span class="n">maxIter</span><span class="o">=</span><span class="mi">5</span><span class="p">,</span> <span class="n">regParam</span><span class="o">=</span><span class="mf">0.01</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">model</span> <span class="o">=</span> <span class="n">lr</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">df</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">test0</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">([</span><span class="n">Row</span><span class="p">(</span><span class="n">features</span><span class="o">=</span><span class="n">Vectors</span><span class="o">.</span><span class="n">dense</span><span class="p">(</span><span class="o">-</span><span class="mf">1.0</span><span class="p">))])</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">test0</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">prediction</span>
<span class="go">0.0</span>
<span class="gp">>>> </span><span class="n">test1</span> <span class="o">=</span> <span class="n">sc</span><span class="o">.</span><span class="n">parallelize</span><span class="p">([</span><span class="n">Row</span><span class="p">(</span><span class="n">features</span><span class="o">=</span><span class="n">Vectors</span><span class="o">.</span><span class="n">sparse</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="p">[</span><span class="mf">1.0</span><span class="p">]))])</span><span class="o">.</span><span class="n">toDF</span><span class="p">()</span>
<span class="gp">>>> </span><span class="k">print</span> <span class="n">model</span><span class="o">.</span><span class="n">transform</span><span class="p">(</span><span class="n">test1</span><span class="p">)</span><span class="o">.</span><span class="n">head</span><span class="p">()</span><span class="o">.</span><span class="n">prediction</span>
<span class="go">1.0</span>
<span class="gp">>>> </span><span class="n">lr</span><span class="o">.</span><span class="n">setParams</span><span class="p">(</span><span class="s">"vector"</span><span class="p">)</span>
<span class="gt">Traceback (most recent call last):</span>
<span class="o">...</span>
<span class="gr">TypeError</span>: <span class="n">Method setParams forces keyword arguments.</span>
</pre></div>
</div>
<dl class="attribute">
<dt id="pyspark.ml.classification.LogisticRegression.featuresCol">
<tt class="descname">featuresCol</tt><em class="property"> = Param(parent=undefined, name='featuresCol', doc='features column name', defaultValue='features')</em><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.featuresCol" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.fit">
<tt class="descname">fit</tt><big>(</big><em>dataset</em>, <em>params={}</em><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.fit" title="Permalink to this definition">¶</a></dt>
<dd><p>Fits a model to the input dataset with optional parameters.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>dataset</strong> – input dataset, which is an instance of
<a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><tt class="xref py py-class docutils literal"><span class="pre">pyspark.sql.DataFrame</span></tt></a></li>
<li><strong>params</strong> – an optional param map that overwrites embedded
params</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">fitted model</p>
</td>
</tr>
</tbody>
</table>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.getFeaturesCol">
<tt class="descname">getFeaturesCol</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.getFeaturesCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of featuresCol or its default value.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.getLabelCol">
<tt class="descname">getLabelCol</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.getLabelCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of labelCol or its default value.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.getMaxIter">
<tt class="descname">getMaxIter</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.getMaxIter" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of maxIter or its default value.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.getPredictionCol">
<tt class="descname">getPredictionCol</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.getPredictionCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of predictionCol or its default value.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.getRegParam">
<tt class="descname">getRegParam</tt><big>(</big><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.getRegParam" title="Permalink to this definition">¶</a></dt>
<dd><p>Gets the value of regParam or its default value.</p>
</dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.classification.LogisticRegression.labelCol">
<tt class="descname">labelCol</tt><em class="property"> = Param(parent=undefined, name='labelCol', doc='label column name', defaultValue='label')</em><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.labelCol" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.classification.LogisticRegression.maxIter">
<tt class="descname">maxIter</tt><em class="property"> = Param(parent=undefined, name='maxIter', doc='max number of iterations', defaultValue=100)</em><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.maxIter" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.classification.LogisticRegression.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt>.</p>
</dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.classification.LogisticRegression.predictionCol">
<tt class="descname">predictionCol</tt><em class="property"> = Param(parent=undefined, name='predictionCol', doc='prediction column name', defaultValue='prediction')</em><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.predictionCol" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="attribute">
<dt id="pyspark.ml.classification.LogisticRegression.regParam">
<tt class="descname">regParam</tt><em class="property"> = Param(parent=undefined, name='regParam', doc='regularization constant', defaultValue=0.1)</em><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.regParam" title="Permalink to this definition">¶</a></dt>
<dd></dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.setFeaturesCol">
<tt class="descname">setFeaturesCol</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.setFeaturesCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.classification.LogisticRegression.featuresCol" title="pyspark.ml.classification.LogisticRegression.featuresCol"><tt class="xref py py-attr docutils literal"><span class="pre">featuresCol</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.setLabelCol">
<tt class="descname">setLabelCol</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.setLabelCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.classification.LogisticRegression.labelCol" title="pyspark.ml.classification.LogisticRegression.labelCol"><tt class="xref py py-attr docutils literal"><span class="pre">labelCol</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.setMaxIter">
<tt class="descname">setMaxIter</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.setMaxIter" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.classification.LogisticRegression.maxIter" title="pyspark.ml.classification.LogisticRegression.maxIter"><tt class="xref py py-attr docutils literal"><span class="pre">maxIter</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.setParams">
<tt class="descname">setParams</tt><big>(</big><em>self</em>, <em>featuresCol="features"</em>, <em>labelCol="label"</em>, <em>predictionCol="prediction"</em>, <em>maxIter=100</em>, <em>regParam=0.1</em><big>)</big><a class="reference internal" href="_modules/pyspark/ml/classification.html#LogisticRegression.setParams"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.setParams" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets params for logistic regression.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.setPredictionCol">
<tt class="descname">setPredictionCol</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.setPredictionCol" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.classification.LogisticRegression.predictionCol" title="pyspark.ml.classification.LogisticRegression.predictionCol"><tt class="xref py py-attr docutils literal"><span class="pre">predictionCol</span></tt></a>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegression.setRegParam">
<tt class="descname">setRegParam</tt><big>(</big><em>value</em><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegression.setRegParam" title="Permalink to this definition">¶</a></dt>
<dd><p>Sets the value of <a class="reference internal" href="#pyspark.ml.classification.LogisticRegression.regParam" title="pyspark.ml.classification.LogisticRegression.regParam"><tt class="xref py py-attr docutils literal"><span class="pre">regParam</span></tt></a>.</p>
</dd></dl>
</dd></dl>
<dl class="class">
<dt id="pyspark.ml.classification.LogisticRegressionModel">
<em class="property">class </em><tt class="descclassname">pyspark.ml.classification.</tt><tt class="descname">LogisticRegressionModel</tt><big>(</big><em>java_model</em><big>)</big><a class="reference internal" href="_modules/pyspark/ml/classification.html#LogisticRegressionModel"><span class="viewcode-link">[source]</span></a><a class="headerlink" href="#pyspark.ml.classification.LogisticRegressionModel" title="Permalink to this definition">¶</a></dt>
<dd><p>Model fitted by LogisticRegression.</p>
<dl class="attribute">
<dt id="pyspark.ml.classification.LogisticRegressionModel.params">
<tt class="descname">params</tt><a class="headerlink" href="#pyspark.ml.classification.LogisticRegressionModel.params" title="Permalink to this definition">¶</a></dt>
<dd><p>Returns all params. The default implementation uses
<tt class="xref py py-func docutils literal"><span class="pre">dir()</span></tt> to get all attributes of type
<tt class="xref py py-class docutils literal"><span class="pre">Param</span></tt>.</p>
</dd></dl>
<dl class="method">
<dt id="pyspark.ml.classification.LogisticRegressionModel.transform">
<tt class="descname">transform</tt><big>(</big><em>dataset</em>, <em>params={}</em><big>)</big><a class="headerlink" href="#pyspark.ml.classification.LogisticRegressionModel.transform" title="Permalink to this definition">¶</a></dt>
<dd><p>Transforms the input dataset with optional parameters.</p>
<table class="docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field-odd field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first simple">
<li><strong>dataset</strong> – input dataset, which is an instance of
<a class="reference internal" href="pyspark.sql.html#pyspark.sql.DataFrame" title="pyspark.sql.DataFrame"><tt class="xref py py-class docutils literal"><span class="pre">pyspark.sql.DataFrame</span></tt></a></li>
<li><strong>params</strong> – an optional param map that overwrites embedded
params</li>
</ul>
</td>
</tr>
<tr class="field-even field"><th class="field-name">Returns:</th><td class="field-body"><p class="first last">transformed dataset</p>
</td>
</tr>
</tbody>
</table>
</dd></dl>
</dd></dl>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="index.html">
<img class="logo" src="_static/spark-logo-hd.png" alt="Logo"/>
</a></p>
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">pyspark.ml package</a><ul>
<li><a class="reference internal" href="#module-pyspark.ml">Module Context</a></li>
<li><a class="reference internal" href="#module-pyspark.ml.feature">pyspark.ml.feature module</a></li>
<li><a class="reference internal" href="#module-pyspark.ml.classification">pyspark.ml.classification module</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="pyspark.streaming.html"
title="previous chapter">pyspark.streaming module</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="pyspark.mllib.html"
title="next chapter">pyspark.mllib package</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/pyspark.ml.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="pyspark.mllib.html" title="pyspark.mllib package"
>next</a></li>
<li class="right" >
<a href="pyspark.streaming.html" title="pyspark.streaming module"
>previous</a> |</li>
<li><a href="index.html">PySpark 1.3.1 documentation</a> »</li>
</ul>
</div>
<div class="footer">
© Copyright .
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.
</div>
</body>
</html>