Skip to content

Commit 891ada5

Browse files
lianchengmarmbrus
authored andcommitted
[SPARK-6696] [SQL] Adds HiveContext.refreshTable to PySpark
<!-- Reviewable:start --> [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/5349) <!-- Reviewable:end --> Author: Cheng Lian <[email protected]> Closes apache#5349 from liancheng/py-refresh-table and squashes the following commits: 004bec0 [Cheng Lian] Adds HiveContext.refreshTable to PySpark
1 parent 7d7384c commit 891ada5

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

python/pyspark/sql/context.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -574,6 +574,15 @@ def _ssql_ctx(self):
574574
def _get_hive_ctx(self):
575575
return self._jvm.HiveContext(self._jsc.sc())
576576

577+
def refreshTable(self, tableName):
578+
"""Invalidate and refresh all the cached the metadata of the given
579+
table. For performance reasons, Spark SQL or the external data source
580+
library it uses might cache certain metadata about a table, such as the
581+
location of blocks. When those change outside of Spark SQL, users should
582+
call this function to invalidate the cache.
583+
"""
584+
self._ssql_ctx.refreshTable(tableName)
585+
577586

578587
class UDFRegistration(object):
579588
"""Wrapper for user-defined function registration."""

0 commit comments

Comments
 (0)