diff options
| author | hushan[胡珊] <hushan@xiaomi.com> | 2015-09-21 14:26:15 -0500 |
|---|---|---|
| committer | Imran Rashid <irashid@cloudera.com> | 2015-09-21 14:26:15 -0500 |
| commit | b78c65b03ae87a3ba348c9d29ff4c296349eb49c (patch) | |
| tree | a80bfbd17a7b7ba4a05496ac62be051df3aff81c /python/pyspark/ml/feature.py | |
| parent | 331f0b10f78a37d96d3e573d211d74a0935265db (diff) | |
| download | spark-b78c65b03ae87a3ba348c9d29ff4c296349eb49c.tar.gz spark-b78c65b03ae87a3ba348c9d29ff4c296349eb49c.tar.bz2 spark-b78c65b03ae87a3ba348c9d29ff4c296349eb49c.zip | |
[SPARK-5259] [CORE] don't submit stage until its dependencies map outputs are registered
Track pending tasks by partition ID instead of Task objects.
Before this change, failure & retry could result in a case where a stage got submitted before the map output from its dependencies get registered. This was due to an error in the condition for registering map outputs.
Author: hushan[胡珊] <hushan@xiaomi.com>
Author: Imran Rashid <irashid@cloudera.com>
Closes #7699 from squito/SPARK-5259.
Diffstat (limited to 'python/pyspark/ml/feature.py')
0 files changed, 0 insertions, 0 deletions