Engine types | Kernel Version | Description |
Presto | SuperSQL-P 1.0 | Based on the native Presto 0.242 version, it supports dynamic loading of data sources, Dymanic Filter enhancement, Iceberg V2 table, INSERT OVERWRITE of non-partitioned tables, and Hive UDF execution. |
SparkSQL | SuperSQL-S 1.0 | It is implemented based on the original Spark3.2 version and supports Iceberg 1.1.0, Hudi 0.12.0 and adaptive Shuffle Manager on this basis. |
| SuperSQL-S 3.5 | It is implemented based on the native Spark3.5 version, and supports Iceberg 1.5.0 and adaptive Shuffle Manager on this basis. The current beta version is backward compatible with various SQL and data governance tasks of SuperSQL-S 1.0, providing a performance improvement of more than 33% over the S1.0 version. |
SparkBatch | Spark 3.5 | It is implemented based on the native Spark3.5 version, and supports Iceberg 1.5.0, Python3 and adaptive Shuffle Manager on this basis. The current beta version is backward compatible with various SQL, jar, pyspark and data governance tasks of Spark 3.2, with a performance improvement of more than 33% over Spark 3.2. |
| Spark 3.2 | It is implemented based on the original Spark3.2 version, which supports Iceberg 1.1.0, Hudi 0.12.0, Python3, and adaptive Shuffle Manager. |
| Spark 2.4 | It is implemented based on the native Spark2.4 version, and supports Iceberg 0.13.1, Python2 and Python3 on this basis. |
Was this page helpful?
You can also Contact sales or Submit a Ticket for help.
Help us improve! Rate your documentation experience in 5 mins.
Feedback