HomePage esProc Function Reference Functions spark_shell() ＜Previous | Next ＞

spark_shell()

Read（204） Label: sql, spark database,

Description:

Execute a SQL command on a Spark table.

Syntax:

spark_shell(con,sql)

Note:

External library function (See External Library Guide).

 

The function executes a SQL command on a Spark table.

Parameter:

con

Database connection string, which can be a Spark connection, a Spark connection using Hudi/Iceberg format, or a Spark connection using Hudi/Iceberg format used with S3.

sql

A SQL statement.

Example:

 

A

 

1

=spark_open("spark.properties")

Connect to a Spark database.

2

=spark_shell(A1,"show tables in mytest1;")

List all tables in the specified database named mytest1.

3

>spark_close(A1)

 

4

=spark_open("hudi.properties")

Connect to a Spark database using Hudi format.

5

=spark_shell(A4,"select * from emp1")

Query data in table emp1.

6

>spark_close(A4)

 

7

=spark_open@i("iceberg.properties")

Connect to a Spark database using Iceberg format.

8

=spark_shell(A7,"drop table test1")

Delete table test1.

9

>spark_close(A7)

 

10

=spark_open("hudi-s3.properties")

Connect to a Spark database using Hudi format used with S3.

11

=spark_shell(A10,"show databases;")

List all databases.

12

>spark_close(A10)

 

13

=spark_open@i("iceberg-s3.properties")

Connect to a Spark database using Iceberg format used with S3.

14

=spark_shell(A12,"select * from icetest1;")

Query data in table icetest1.

15

>spark_close(A12)