Global temporary view

Global temporary views are primarily used to share within a session and then get automatically dropped once the session ends. These session-scoped views also serve as a temporary table on which SQL queries can be made and are stored in database global_temp. The database name can be changed by modifying the SQL configuration spark.sql.globalTempDatabase. However, qualified names should be used to access GlobalTempView(database.tableName) or else it throws an error Table or view not found:

deptDf.createGlobalTempView("dept_global_view");
sparkSession.newSession().sql("SELECT deptno,dname,loc, rank() OVER (PARTITION BY loc ORDER BY deptno ) FROM global_temp.dept_global_view").show();

While using Thrift Server with a multi-session (spark.sql.hive.thriftServer.singleSession = false), a global temporary view can be used to access data from a different session. Prior to Spark 2.1, the only way to share data across sessions was to persist the data and then access it using catalog.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
13.59.48.161