gen22 release notes

Changelog

Gen22 is a bugfix release to correct a build issue that affected 3.4.1 and 3.5 images in gen21.

Available tags

platform-3.4.1-hadoop-3.3.4-java-11-scala-2.12-python-3.10-gen22

Additional tags

• platform-3.4-gen22
• platform-3.4.1-gen22

Dependency versions

name	version
Spark	3.4.1
Hadoop	3.3.4
Java	11
Scala	2.12.17
Python	3.10.10
Hive	2.3.9
Pyarrow	11.0.0

Supported connectors

name	version
S3 (s3a:// or s3://)	AWS 1.12.262
GCS (gs://)	2.1.5
ADLS gen1 (adl://)	ADLS SDK 2.3.9
ADLS gen2 (abfss://)	Azure Storage 7.0.1
Azure Blob Storage (wasbs://)	Azure Storage 7.0.1
AWS Glue	Hive 2.3.9

Additional formats

name	version
Delta	2.4.0
Snowflake	Not supported

platform-3.5.0-hadoop-3.3.4-java-11-scala-2.12-python-3.10-gen22

Additional tags

• platform-3.5-gen22
• platform-3.5.0-gen22

Dependency versions

name	version
Spark	3.5.0
Hadoop	3.3.4
Java	11
Scala	2.12.18
Python	3.10.10
Hive	2.3.9
Pyarrow	11.0.0

Supported connectors

name	version
S3 (s3a:// or s3://)	AWS 1.12.262
GCS (gs://)	2.1.5
ADLS gen1 (adl://)	ADLS SDK 2.3.9
ADLS gen2 (abfss://)	Azure Storage 7.0.1
Azure Blob Storage (wasbs://)	Azure Storage 7.0.1
AWS Glue	Hive 2.3.9

Additional formats

name	version
Delta	0.8.0
Snowflake	Not supported