spark_query()

con

Database connection string, which can be a normal Spark connection, a Spark connection using Hudi/Iceberg format, or a Spark connection using Hudi/Iceberg format used with S3.

sql

A SQL statement, like select * from table.

tableName

A table name (query the table data using file query methods), which forms a url along with parameter warehouse in configuration file .properties, such as hdfs://localhost:9000/user/hive/warehouse/tableName,  which shows the location of the table in HDFS.

@c

Return a cursor.

@d

Do not display the automatically generated field for a Hudi table; only invalid for Hudi tables.

A

1

=spark_open("spark.properties")

Connect to a Spark database.

2

=spark_query(A1,"select * from test1")

Query data using a SQL statement.

3

>spark_close(A1)

4

=spark_open("hudi.properties")

Connect to a Spark database using Hudi format.

5

=spark_query@d(A4,"select * from huditb1")

Do not display the automatically generated field for the Hudi table in the returned result set.

6

>spark_close(A4)

7

=spark_open@i("iceberg.properties")

Connect to a Spark database using Iceberg format.

8

=spark_query(A7,"icetest1")

Query data using a file query method, during which the table name forms a url along with parameter warehouse in configuration file , such as hdfs://localhost:9000/user/hive/warehouse/icetest1.

9

>spark_close(A7)

10

=spark_open("hudi-s3.properties")

Connect to a Spark database using Hudi format used with S3.

11

=spark_query@c(A10,"select * from emp1")

Query data and return a cursor.

12

>spark_close(A10)

13

=spark_open("iceberg-s3.properties")

Connect to a Spark database using Iceberg format used with S3.

14

=spark_query(A13,"select * from test1")

Query data using a SQL statement.

15

>spark_close(A13)