Support optional DuckDB-backed inputs - #289
Conversation
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
|
@copilot in unit test, convert the example data into duckdb and run estimate, and make sure the results are the same as with normal df |
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
Added a DuckDB regression test in commit |
| sccomp_estimate.tbl_duckdb_connection <- function(.data, ...) { | ||
| check_and_install_packages(c("duckdb", "dbplyr")) | ||
|
|
||
| sccomp_estimate.data.frame(dplyr::collect(.data), ...) |
There was a problem hiding this comment.
you should pass the dataset through until you reach the |> count(...) and then you can collect after the count
There was a problem hiding this comment.
Updated in commit fbe02f8: DuckDB-backed data now remains lazy through the existing sample/cell-group count() checks and is collected afterward, before the remaining data-frame modeling steps.
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
| #' @keywords internal | ||
| #' @noRd | ||
| make_rectangular_data = function(.data, .sample, .cell_group, .count, formula_composition) { | ||
| make_rectangular_data = function(.data, .sample, .cell_group, .count, formula_composition, collect_after_count = FALSE) { |
There was a problem hiding this comment.
instead of collect_after_count, you can just check is(...) so you avoid creting too many arguments.
There was a problem hiding this comment.
Applied in commit 4bdacca: removed collect_after_count and now detect tbl_duckdb_connection directly in make_rectangular_data() before collecting.
Co-authored-by: stemangiola <7232890+stemangiola@users.noreply.github.com>
sccomp_estimate()could not dispatch on DuckDB-backed lazy tables because no matching S3 method existed. DuckDB should remain optional rather than becoming a hard package dependency.duckdbanddbplyrtoSuggests.sccomp_estimate.tbl_duckdb_connection().dplyr::collect(), and delegate to the existing data-frame implementation.