diff options
| author | Brennon York <brennon.york@capitalone.com> | 2015-03-26 19:08:09 -0700 |
|---|---|---|
| committer | Ankur Dave <ankurdave@gmail.com> | 2015-03-26 19:08:09 -0700 |
| commit | 39fb57968352549f2276ac4fcd2b92988ed6fe42 (patch) | |
| tree | 811c1ecc0e813d299cdfdac6f1d6521883beb4b8 /project | |
| parent | aad00322765d6041e817a6bd3fcff2187d212057 (diff) | |
| download | spark-39fb57968352549f2276ac4fcd2b92988ed6fe42.tar.gz spark-39fb57968352549f2276ac4fcd2b92988ed6fe42.tar.bz2 spark-39fb57968352549f2276ac4fcd2b92988ed6fe42.zip | |
[SPARK-6510][GraphX]: Add Graph#minus method to act as Set#difference
Adds a `Graph#minus` method which will return only unique `VertexId`'s from the calling `VertexRDD`.
To demonstrate a basic example with pseudocode:
```
Set((0L,0),(1L,1)).minus(Set((1L,1),(2L,2)))
> Set((0L,0))
```
Author: Brennon York <brennon.york@capitalone.com>
Closes #5175 from brennonyork/SPARK-6510 and squashes the following commits:
248d5c8 [Brennon York] added minus(VertexRDD[VD]) method to avoid createUsingIndex and updated the mask operations to simplify with andNot call
3fb7cce [Brennon York] updated graphx doc to reflect the addition of minus method
6575d92 [Brennon York] updated mima exclude
aaa030b [Brennon York] completed graph#minus functionality
7227c0f [Brennon York] beginning work on minus functionality
Diffstat (limited to 'project')
| -rw-r--r-- | project/MimaExcludes.scala | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 56f5dbe53f..b9f40046e1 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -51,6 +51,9 @@ object MimaExcludes { "org.apache.spark.broadcast.HttpBroadcastFactory.newBroadcast"), ProblemFilters.exclude[IncompatibleResultTypeProblem]( "org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast") + ) ++ Seq( + // SPARK-6510 Add a Graph#minus method acting as Set#difference + ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.graphx.VertexRDD.minus") ) case v if v.startsWith("1.3") => |