v0.15.6

🐛 Bug Fixes

• Always deep-copy Parquet configs by @clairemcginty in #5920

📗 Documentation

• Add deprecation header by @kellen in #5882

Full Changelog: v0.15.5...v0.15.6

v0.15.5

Scio 0.15.5 includes Beam 2.73

🚀 Enhancements

• Make Voyager index and names filenames configurable by @dylanrb123 in #5914

🌱 Dependency Updates

• Upgrade to Beam 2.72 by @cjordn in #5905
• Update sbt-avro, sbt-avro-compiler-bridge to 4.0.2 by @clairemcginty in #5889
• Update mysql-connector-j to 9.6.0 by @clairemcginty in #5893
• Upgrade SBT to 1.12.7 by @clairemcginty in #5906
• Update sbt-typelevel to 0.8.5 by @clairemcginty in #5915
• Update jedis to 7.2.1 by @clairemcginty in #5917
• Update sbt-scoverage to 2.4.4 by @clairemcginty in #5913
• Update scalafmt-core to 3.10.7 by @clairemcginty in #5911
• Update scalac-options to 0.1.9 by @clairemcginty in #5916
• Update munit to 1.2.4 by @clairemcginty in #5910
• Update scalactic to 3.2.20 by @clairemcginty in #5909
• Update sbt, sbt-dependency-tree, ... to 1.12.8 by @clairemcginty in #5908
• Update scalatest to 3.2.20 by @clairemcginty in #5912
• Bump sparkey version by @spkrka in #5902

Full Changelog: v0.15.4...v0.15.5

v0.15.4

🚀 Enhancements

• Support Java 25 by @clairemcginty in #5883

🏗️ Build Improvements

• Add test to ParquetTestUtilsTest to check list projections and CaseMappers by @clairemcginty in #5900

🌱 Dependency Updates

• Update elasticsearch-java to 8.19.13 by @clairemcginty in #5888
• Update sbt-bom to 1.0.33 by @clairemcginty in #5891
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.28.2 by @clairemcginty in #5890
• Update neo4j-java-driver to 4.4.22 by @clairemcginty in #5898
• Update parquet-avro, parquet-column, ... to 1.17.0 by @clairemcginty in #5897
• Update mssql-jdbc to 13.4.0.jre11 by @clairemcginty in #5892
• Update metrics-core to 4.2.38 by @clairemcginty in #5895
• Update iceberg-common, iceberg-core, ... to 1.10.1 by @clairemcginty in #5896
• Upgrade Sparkey to 3.5.1 by @clairemcginty in #5899

Full Changelog: v0.15.3...v0.15.4

v0.15.3

Scio 0.15.3 includes Beam 2.71

🐛 Bug Fixes

• (fix #5874) Fix missing field descriptions for typed BigQuery writes by @clairemcginty in #5875
• Rename BufferedExternalSorter to avoid class load conflicts by @shnapz in #5877
• fix dropped AvroType @doc annotations by @clairemcginty in #5879
• Fix record-level Avro doc by @clairemcginty in #5880

🌱 Dependency Updates

• Upgrade to Beam 2.71 by @clairemcginty in #5873
• Use slf4j v2 by @kellen in #5872
• Bump Magnolify to 0.9.3 by @clairemcginty in #5878

Full Changelog: v0.15.2...v0.15.3

v0.15.2

Scio 0.15.2 includes Beam 2.70

🚀 Enhancements

• Enable Beam lineage for JdbcIO by @shnapz in #5849
• Make SMB classes extensible for external libraries by @spkrka in #5869

🐛 Bug Fixes

• Fix SmbIO tap to use FileOperations coder by @spkrka in #5870

🌱 Dependency Updates

• Upgrade to Beam 2.70 by @clairemcginty in #5867
• Update jedis to 7.2.0 by @scala-steward in #5863
• Update sbt-typelevel to 0.8.4 by @scala-steward in #5862
• Update scalafmt-core to 3.10.3 by @scala-steward in #5860
• Update sbt-mdoc to 2.8.2 by @scala-steward in #5859
• Update scala-compiler, scala-library, ... to 2.13.18 by @scala-steward in #5857
• Update scalacheck to 1.19.0 by @scala-steward in #5858
• Update caffeine to 3.2.3 by @scala-steward in #5853
• Update elasticsearch-java to 8.19.9 by @scala-steward in #5851
• Update tensorflow-core-native to 1.1.0 by @scala-steward in #5861
• Update beam-examples-java, ... to 2.69.0 by @clairemcginty in #5806
• Update sbt, sbt-dependency-tree, ... to 1.12.0 by @scala-steward in #5865
• Update sbt-unidoc to 0.6.1 by @scala-steward in #5864
• Update scala-compiler, scala-library, ... to 2.12.21 by @scala-steward in #5856
• Bump checkout/download-artifact/iceberg/testcontainers by @kellen in #5871
• Prepare for 0.15.2 release by @clairemcginty in #5868

Full Changelog: v0.15.1...v0.15.2

v0.15.1

Bugfix release

• Un-deprecate macro annotations to avoid magnolify serialization issues, deprecate typed avro and BQ methods by @kellen in #5846
• Check if org.apache.avro.SERIALIZABLE_CLASSES in scio-smb Parquet writes by @clairemcginty in #5847
• Null check on org.apache.avro.SERIALIZABLE_CLASSES prop by @clairemcginty in #5829

🚀 Enhancements

• scalafix for 0.15 by @kellen in #5822

📗 Documentation

• Add migration guide, deprecations for 0.15 by @kellen in #5845

🌱 Dependency Updates

• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.27.1 by @clairemcginty in #5833
• Update sbt-bom to 1.0.31 by @clairemcginty in #5834
• Update pprint to 0.9.6 by @clairemcginty in #5835
• Update sparkey to 3.5.0 by @clairemcginty in #5836
• Update scalacheck to 1.18.1 by @clairemcginty in #5840
• Update sbt-scoverage to 2.4.3 by @clairemcginty in #5843
• Update sbt-scalafix to 0.14.5 by @clairemcginty in #5830
• Upgrade Sparkey to 3.5.0 by @spkrka in #5827

Full Changelog: v0.15.0...v0.15.1

v0.15.0

Scio 0.15 adds support for Iceberg, beam's managed transforms, and Avro 1.11's java-class annotations.

🚀 Enhancements

• Magnolify API by @kellen in #5286
• (fix #5067) Drop Java 8 Targeting by @clairemcginty in #5203
• Managed + Iceberg IO by @kellen in #5494
• Support writing extra metadata in scio-parquet by @clairemcginty in #5411
• Update IcebergIO read/write parameters by @kellen in #5823
• Add integration test for Magnolify TableRowType compatibility by @clairemcginty in #5818
• Support Avro 1.11 java-class annotations in scio-parquet by @clairemcginty in #5821
• Add ParquetMetadataDoFn by @kellen in #5820
• v0.15.0 by @kellen in #5609
• Remove stale comments, make some previously-deprecated methods private by @kellen in #5584

🏗️ Build Improvements

• Vendor Socco by @kellen in #5811
• Exclude dependabot from updating ci.yml by @kellen in #5813
• Move CI env definition by @kellen in #5826
• Remove developers list by @kellen in #5548
• Fix BT integration tests by @kellen in #5606
• Fix BT integration tests mark ii by @kellen in #5608
• Add headers, remove unused by @kellen in #5619

🐛 Bug Fixes

• Fix Iceberg integration test, defaults by @kellen in #5825
• Fix seed setting in samplers by @kellen in #5816
• Add check for java 17 SCollectionMatcher (fix #5483) by @kellen in #5605

🌱 Dependency Updates

• Upgrade to Avro 1.11.5 by @clairemcginty in #5799
• Update sbt to 1.11.7, sbt-mdoc to 2.8.0, sbt-java-formatter to 0.10.0… by @kellen in #5815
• Bump to magnolify 0.9.2 by @kellen in #5824
• Update scala-compiler, scala-library, ... to 2.13.17 by @clairemcginty in #5786
• Bump coursier/cache-action from 6 to 7 by @dependabot[bot] in #5797
• Update sbt-typelevel to 0.8.2 by @clairemcginty in #5808
• Update sbt-jmh to 0.4.8 by @clairemcginty in #5809
• Update mysql-connector-j to 9.5.0 by @clairemcginty in #5804
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.27.0 by @clairemcginty in #5802
• Update jedis to 7.0.0 by @clairemcginty in #5810
• Update sbt-scoverage to 2.4.1 by @clairemcginty in #5807
• Update sbt-bom to 1.0.30 by @clairemcginty in #5803
• Update testcontainers-scala-elasticsearch, ... to 0.43.6 by @clairemcginty in #5801
• Update elasticsearch-java to 8.19.7 by @clairemcginty in #5800
• Update magnolify to 0.8.0, tensorflow to 1.0.0, caffeine to 3.2.0 by @kellen in #5604
• magnolify 0.9 by @kellen in #5812

Full Changelog: v0.14.20...v0.15.0

v0.14.20

This release includes Beam 2.68.0 support. It also upgrades the major version of protobuf-java to 4.x.

🐛 Bug Fixes

• Fix serialization error when sortMergeCoGroup has secondary keys by @charles-tan in #5768

🏗️ Build Improvements

• Add test assertion for OverrideTypeProvider field name by @clairemcginty in #5766
• Fix mergeStrategy for scio-repl by @clairemcginty in #5794

🌱 Dependency Updates

• Upgrade to Beam 2.68 by @clairemcginty in #5769
• Upgrade protobuf-java and protoc to 4.x by @clairemcginty in #5701
• Update parquet-avro, parquet-column, ... to 1.16.0 by @clairemcginty in #5762
• Update jedis to 6.2.0 by @clairemcginty in #5763
• Update elasticsearch-java to 8.19.4 by @clairemcginty in #5770
• Update sbt, sbt-dependency-tree, ... to 1.11.6 by @clairemcginty in #5778
• Update jna to 5.18.1 by @clairemcginty in #5776
• Update scalafmt-core to 3.10.1 by @clairemcginty in #5791
• Update munit to 1.2.1 by @clairemcginty in #5789
• Update sbt-scalafix to 0.14.4 by @clairemcginty in #5782
• Update elasticsearch-java to 8.19.5 by @clairemcginty in #5783
• Update circe-core, circe-generic, ... to 0.14.15 by @clairemcginty in #5774
• Update metrics-core to 4.2.37 by @clairemcginty in #5775
• Update sbt-ghpages to 0.9.0 by @clairemcginty in #5771
• Update sbt-bom to 1.0.29 by @clairemcginty in #5772
• Bump netty to 4.1.128 by @clairemcginty in #5793

Full Changelog: v0.14.19...v0.14.20

v0.14.19

v0.14.19

This release includes Beam 2.67.0 support.

🐛 Bug Fixes

🚀 Enhancements

• Add option to enable flow control for BigTable writes by @clairemcginty in #5747
• Unblock usage of gcs-connector 3.x by @clairemcginty in #5730
• Add experimental support for vectoredRead-compatible stream in scio-parquet by @clairemcginty in #5732

🏗️ Build Improvements

• Clean unused MiMa filters by @clairemcginty in #5729
• Bump actions/setup-java from 4 to 5 by @dependabot[bot] in #5748
• Bump actions/download-artifact from 3 to 5 by @dependabot[bot] in #5733

🌱 Dependency Updates

• Upgrade Beam to 2.67 by @clairemcginty in #5735
• Update sbt, sbt-dependency-tree, ... to 1.11.5 by @clairemcginty in #5752
• Update sbt-scalafix to 0.14.3 by @clairemcginty in #5685
• Update sbt-bom to 1.0.27 by @clairemcginty in #5738
• Update sbt-unidoc to 0.6.0 by @clairemcginty in #5750
• Update fansi to 0.5.1 by @clairemcginty in #5739
• Update mssql-jdbc to 13.2.0.jre11 by @clairemcginty in #5741
• Update mysql-connector-j to 9.4.0 by @clairemcginty in #5742
• Update sbt, sbt-dependency-tree, ... to 1.11.4 by @clairemcginty in #5744
• Update jedis to 6.1.0 by @clairemcginty in #5746
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.25.3 by @clairemcginty in #5737
• Update elasticsearch-java to 8.19.2 by @clairemcginty in #5749
• Update scalafmt-core to 3.9.9 by @clairemcginty in #5745

Full Changelog: v0.14.18...v0.14.19

v0.14.18

v0.14.18

This release includes Beam 2.66.0 support.

🐛 Bug Fixes

• (fix #5713) Fix BigQueryType#fromAvro on missing fields for Avro 1.11 by @clairemcginty in #5714

🚀 Enhancements

• Enable deterministic sampling with SCollection.sample by @crystall-spotify in #5709

📗 Documentation

• Update release docs by @clairemcginty in #5698
• Update Apache Beam version matrix in documentation by @charles-tan in #5727

🏗️ Build Improvements

• Allow manual trigger of integration tests in CI by @clairemcginty in #5706
• Warm up BQ service calls in IT tests by @clairemcginty in #5708
• Use unique name for manually triggered CI by @clairemcginty in #5707

🌱 Dependency Updates

• Bump beam to 2.66.0 by @charles-tan in #5715
• Update metrics-core to 4.2.32 by @clairemcginty in #5689
• Update cloud-sql-connector-jdbc-sqlserver, ... to 1.25.1 by @clairemcginty in #5688
• Update sbt-avro, sbt-avro-compiler-bridge to 4.0.1 by @clairemcginty in #5687
• Update sbt, sbt-dependency-tree, ... to 1.11.3 by @clairemcginty in #5722
• Update circe-core, circe-generic, ... to 0.14.14 by @clairemcginty in #5711
• Update sbt-protoc to 1.0.8 by @clairemcginty in #5710
• Update sonatype host to central portal by @clairemcginty in #5717
• Update scalafmt-core to 3.9.8 by @clairemcginty in #5724
• Update sbt-mdoc to 2.7.2 by @clairemcginty in #5723
• Update mssql-jdbc to 12.10.1.jre11 by @clairemcginty in #5720
• Update elasticsearch-java to 8.18.3 by @clairemcginty in #5719

New Contributors

• @crystall-spotify made their first contribution in #5709

Full Changelog: v0.14.17...v0.14.18