diff options
author | Tathagata Das <tathagata.das1565@gmail.com> | 2015-08-14 15:54:14 -0700 |
---|---|---|
committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-08-14 15:54:14 -0700 |
commit | 18a761ef7a01a4dfa1dd91abe78cd68f2f8fdb67 (patch) | |
tree | 5d1ed89c47ba788b659a931cdab54788fbafe8a9 /sql/catalyst | |
parent | f3bfb711c1742d0915e43bda8230b4d1d22b4190 (diff) | |
download | spark-18a761ef7a01a4dfa1dd91abe78cd68f2f8fdb67.tar.gz spark-18a761ef7a01a4dfa1dd91abe78cd68f2f8fdb67.tar.bz2 spark-18a761ef7a01a4dfa1dd91abe78cd68f2f8fdb67.zip |
[SPARK-9968] [STREAMING] Reduced time spent within synchronized block to prevent lock starvation
When the rate limiter is actually limiting the rate at which data is inserted into the buffer, the synchronized block of BlockGenerator.addData stays blocked for long time. This causes the thread switching the buffer and generating blocks (synchronized with addData) to starve and not generate blocks for seconds. The correct solution is to not block on the rate limiter within the synchronized block for adding data to the buffer.
Author: Tathagata Das <tathagata.das1565@gmail.com>
Closes #8204 from tdas/SPARK-9968 and squashes the following commits:
8cbcc1b [Tathagata Das] Removed unused val
a73b645 [Tathagata Das] Reduced time spent within synchronized block
Diffstat (limited to 'sql/catalyst')
0 files changed, 0 insertions, 0 deletions