diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-04-14 10:58:06 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2016-04-14 10:58:06 -0700 |
| commit | 1d04c86fc575470e15f6667076377cea102552d7 (patch) | |
| tree | 0c0b2cc924dcaf4099198e17b352a04183c9b0c1 /python | |
| parent | a46f98d3f4ba6a79f4ef789806fec80a7d4f342d (diff) | |
| download | spark-1d04c86fc575470e15f6667076377cea102552d7.tar.gz spark-1d04c86fc575470e15f6667076377cea102552d7.tar.bz2 spark-1d04c86fc575470e15f6667076377cea102552d7.zip | |
[SPARK-14558][CORE] In ClosureCleaner, clean the outer pointer if it's a REPL line object
## What changes were proposed in this pull request?
When we clean a closure, if its outermost parent is not a closure, we won't clone and clean it as cloning user's objects is dangerous. However, if it's a REPL line object, which may carry a lot of unnecessary references(like hadoop conf, spark conf, etc.), we should clean it as it's not a user object.
This PR improves the check for user's objects to exclude REPL line object.
## How was this patch tested?
existing tests.
Author: Wenchen Fan <wenchen@databricks.com>
Closes #12327 from cloud-fan/closure.
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions